Oct 30 21:39:45.003242 ip-10-0-140-18 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Oct 30 21:39:45.003257 ip-10-0-140-18 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Oct 30 21:39:45.003267 ip-10-0-140-18 systemd[1]: kubelet.service: Failed with result 'resources'.
Oct 30 21:39:45.003597 ip-10-0-140-18 systemd[1]: Failed to start Kubernetes Kubelet.
Oct 30 21:39:55.066737 ip-10-0-140-18 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Oct 30 21:39:55.066752 ip-10-0-140-18 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot ee1b8de9269240049fb20c9e0790d8fd --
Oct 30 21:41:51.206635 ip-10-0-140-18 systemd[1]: Starting Kubernetes Kubelet...
Oct 30 21:41:51.556839 ip-10-0-140-18 kubenswrapper[2580]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 21:41:51.556839 ip-10-0-140-18 kubenswrapper[2580]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Oct 30 21:41:51.556839 ip-10-0-140-18 kubenswrapper[2580]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 21:41:51.556839 ip-10-0-140-18 kubenswrapper[2580]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 21:41:51.560507 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.560414    2580 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:41:51.560507 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.560505    2580 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:41:51.560507 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.560512    2580 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:41:51.560724 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.560516    2580 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:41:51.560724 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.560521    2580 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:41:51.560724 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.560526    2580 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:41:51.560724 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.560529    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:41:51.560724 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.560533    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:41:51.560724 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.560537    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:41:51.560724 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.560541    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:41:51.560724 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.560544    2580 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:41:51.560724 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.560549    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:41:51.560724 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.560552    2580 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:41:51.560724 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.560557    2580 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:41:51.560724 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.560560    2580 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:41:51.560724 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.560564    2580 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:41:51.560724 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.560568    2580 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:41:51.560724 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.560581    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:41:51.560724 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.560586    2580 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:41:51.560724 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.560590    2580 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:41:51.560724 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.560594    2580 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:41:51.560724 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.560597    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:41:51.560724 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.560601    2580 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:41:51.561398 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.560605    2580 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:41:51.561398 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.560609    2580 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:41:51.561398 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.560613    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:41:51.561398 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.560617    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:41:51.561398 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.560621    2580 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:41:51.561398 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.560624    2580 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:41:51.561398 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.560628    2580 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:41:51.561398 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.560632    2580 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:41:51.561398 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.560636    2580 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:41:51.561398 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.560640    2580 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:41:51.561398 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.560645    2580 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:41:51.561398 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.560650    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:41:51.561398 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.560654    2580 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:41:51.561398 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.560659    2580 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:41:51.561398 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.560663    2580 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:41:51.561398 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.560667    2580 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:41:51.561398 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.560672    2580 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:41:51.561398 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.560676    2580 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:41:51.561398 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.560681    2580 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:41:51.561398 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.560685    2580 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:41:51.561898 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.560690    2580 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:41:51.561898 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.560694    2580 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:41:51.561898 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.560699    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:41:51.561898 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.560703    2580 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:41:51.561898 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.560708    2580 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:41:51.561898 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.560713    2580 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:41:51.561898 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.560718    2580 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:41:51.561898 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.560723    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:41:51.561898 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.560727    2580 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:41:51.561898 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.560731    2580 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:41:51.561898 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.560735    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:41:51.561898 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.560740    2580 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:41:51.561898 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.560746    2580 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:41:51.561898 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.560754    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:41:51.561898 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.560759    2580 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:41:51.561898 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.560765    2580 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:41:51.561898 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.560770    2580 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:41:51.561898 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.560775    2580 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:41:51.561898 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.560780    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:41:51.562367 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.560784    2580 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:41:51.562367 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.560788    2580 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:41:51.562367 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.560793    2580 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:41:51.562367 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.560797    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:41:51.562367 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.560802    2580 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:41:51.562367 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.560806    2580 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:41:51.562367 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.560811    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:41:51.562367 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.560816    2580 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:41:51.562367 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.560822    2580 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:41:51.562367 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.560826    2580 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:41:51.562367 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.560830    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:41:51.562367 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.560834    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:41:51.562367 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.560839    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:41:51.562367 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.560843    2580 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:41:51.562367 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.560848    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:41:51.562367 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.560853    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:41:51.562367 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.560857    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:41:51.562367 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.560861    2580 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:41:51.562367 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.560866    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:41:51.562367 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.560871    2580 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:41:51.562850 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.560875    2580 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:41:51.562850 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.560880    2580 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:41:51.562850 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.560885    2580 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:41:51.562850 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.560889    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:41:51.562850 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.560893    2580 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:41:51.562850 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.560898    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:41:51.562850 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.560902    2580 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:41:51.562850 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.560907    2580 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:41:51.562850 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.560911    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:41:51.562850 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.560915    2580 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:41:51.562850 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.560919    2580 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:41:51.562850 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.560924    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:41:51.562850 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.560928    2580 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:41:51.562850 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.560932    2580 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:41:51.562850 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.560937    2580 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:41:51.562850 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.560941    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:41:51.562850 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.560946    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:41:51.562850 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.560952    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:41:51.562850 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.560957    2580 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:41:51.563623 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.560964    2580 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:41:51.563623 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.560968    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:41:51.563623 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.560972    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:41:51.563623 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.560976    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:41:51.563623 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.560981    2580 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:41:51.564057 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.564039    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:41:51.564057 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.564055    2580 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:41:51.564177 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.564061    2580 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:41:51.564177 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.564066    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:41:51.564177 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.564071    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:41:51.564177 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.564075    2580 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:41:51.564177 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.564079    2580 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:41:51.564177 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.564084    2580 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:41:51.564177 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.564088    2580 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:41:51.564177 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.564096    2580 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:41:51.564177 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.564102    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:41:51.564177 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.564106    2580 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:41:51.564177 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.564110    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:41:51.564177 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.564115    2580 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:41:51.564177 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.564119    2580 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:41:51.564177 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.564123    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:41:51.564177 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.564127    2580 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:41:51.564177 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.564131    2580 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:41:51.564177 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.564135    2580 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:41:51.564177 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.564140    2580 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:41:51.564177 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.564144    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:41:51.565053 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.564149    2580 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:41:51.565053 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.564153    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:41:51.565053 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.564157    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:41:51.565053 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.564162    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:41:51.565053 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.564166    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:41:51.565053 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.564171    2580 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:41:51.565053 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.564175    2580 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:41:51.565053 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.564179    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:41:51.565053 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.564184    2580 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:41:51.565053 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.564187    2580 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:41:51.565053 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.564192    2580 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:41:51.565053 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.564196    2580 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:41:51.565053 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.564200    2580 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:41:51.565053 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.564204    2580 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:41:51.565053 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.564208    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:41:51.565053 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.564212    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:41:51.565053 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.564217    2580 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:41:51.565053 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.564221    2580 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:41:51.565053 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.564233    2580 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:41:51.565053 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.564238    2580 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:41:51.565923 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.564243    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:41:51.565923 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.564247    2580 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:41:51.565923 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.564251    2580 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:41:51.565923 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.564255    2580 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:41:51.565923 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.564259    2580 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:41:51.565923 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.564263    2580 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:41:51.565923 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.564267    2580 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:41:51.565923 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.564271    2580 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:41:51.565923 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.564276    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:41:51.565923 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.564280    2580 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:41:51.565923 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.564284    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:41:51.565923 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.564288    2580 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:41:51.565923 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.564293    2580 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:41:51.565923 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.564313    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:41:51.565923 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.564318    2580 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:41:51.565923 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.564323    2580 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:41:51.565923 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.564327    2580 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:41:51.565923 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.564331    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:41:51.565923 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.564336    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:41:51.565923 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.564340    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:41:51.566824 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.564344    2580 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:41:51.566824 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.564348    2580 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:41:51.566824 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.564355    2580 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:41:51.566824 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.564361    2580 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:41:51.566824 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.564366    2580 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:41:51.566824 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.564371    2580 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:41:51.566824 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.564375    2580 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:41:51.566824 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.564379    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:41:51.566824 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.564393    2580 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:41:51.566824 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.564397    2580 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:41:51.566824 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.564402    2580 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:41:51.566824 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.564406    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:41:51.566824 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.564410    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:41:51.566824 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.564416    2580 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:41:51.566824 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.564421    2580 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:41:51.566824 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.564425    2580 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:41:51.566824 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.564429    2580 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:41:51.566824 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.564434    2580 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:41:51.566824 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.564439    2580 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:41:51.567645 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.564443    2580 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:41:51.567645 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.564447    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:41:51.567645 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.564451    2580 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:41:51.567645 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.564455    2580 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:41:51.567645 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.564460    2580 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:41:51.567645 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.564464    2580 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:41:51.567645 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.564469    2580 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:41:51.567645 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.564474    2580 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:41:51.567645 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.564478    2580 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:41:51.567645 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.564482    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:41:51.567645 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.564486    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:41:51.567645 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.564491    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:41:51.567645 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.564494    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:41:51.567645 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.564498    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:41:51.567645 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.564503    2580 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:41:51.567645 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.564507    2580 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:41:51.567645 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.564511    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:41:51.567645 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.564515    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:41:51.567645 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.564519    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:41:51.568464 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.564524    2580 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:41:51.568464 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.564530    2580 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:41:51.568464 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.564535    2580 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:41:51.568464 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.564539    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:41:51.568464 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.564544    2580 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:41:51.568464 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.564548    2580 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:41:51.568464 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.564552    2580 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:41:51.568464 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.565475    2580 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:41:51.568464 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.565485    2580 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:41:51.568464 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.565490    2580 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:41:51.568464 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.565495    2580 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:41:51.568464 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.565499    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:41:51.568464 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.565503    2580 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:41:51.568464 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.565508    2580 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:41:51.568464 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.565512    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:41:51.568464 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.565516    2580 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:41:51.568464 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.565521    2580 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:41:51.568464 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.565525    2580 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:41:51.568464 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.565530    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:41:51.568931 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.565535    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:41:51.568931 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.565540    2580 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:41:51.568931 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.565544    2580 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:41:51.568931 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.565548    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:41:51.568931 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.565552    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:41:51.568931 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.565556    2580 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:41:51.568931 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.565560    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:41:51.568931 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.565564    2580 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:41:51.568931 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.565569    2580 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:41:51.568931 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.565573    2580 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:41:51.568931 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.565577    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:41:51.568931 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.565581    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:41:51.568931 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.565585    2580 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:41:51.568931 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.565590    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:41:51.568931 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.565594    2580 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:41:51.568931 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.565598    2580 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:41:51.568931 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.565603    2580 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:41:51.568931 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.565607    2580 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:41:51.568931 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.565611    2580 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:41:51.569543 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.565615    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:41:51.569543 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.565620    2580 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:41:51.569543 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.565625    2580 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:41:51.569543 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.565629    2580 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:41:51.569543 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.565633    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:41:51.569543 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.565638    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:41:51.569543 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.565642    2580 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:41:51.569543 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.565647    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:41:51.569543 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.565652    2580 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:41:51.569543 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.565656    2580 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:41:51.569543 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.565660    2580 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:41:51.569543 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.565665    2580 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:41:51.569543 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.565669    2580 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:41:51.569543 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.565673    2580 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:41:51.569543 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.565678    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:41:51.569543 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.565682    2580 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:41:51.569543 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.565686    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:41:51.569543 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.565690    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:41:51.569543 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.565696    2580 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:41:51.570093 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.565700    2580 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:41:51.570093 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.565704    2580 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:41:51.570093 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.565708    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:41:51.570093 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.565712    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:41:51.570093 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.565716    2580 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:41:51.570093 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.565720    2580 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:41:51.570093 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.565724    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:41:51.570093 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.565729    2580 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:41:51.570093 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.565733    2580 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:41:51.570093 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.565737    2580 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:41:51.570093 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.565741    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:41:51.570093 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.565745    2580 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:41:51.570093 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.565750    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:41:51.570093 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.565754    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:41:51.570093 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.565758    2580 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:41:51.570093 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.565762    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:41:51.570093 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.565767    2580 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:41:51.570093 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.565775    2580 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:41:51.570093 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.565781    2580 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:41:51.570580 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.565786    2580 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:41:51.570580 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.565791    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:41:51.570580 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.565795    2580 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:41:51.570580 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.565799    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:41:51.570580 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.565803    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:41:51.570580 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.565807    2580 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:41:51.570580 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.565812    2580 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:41:51.570580 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.565816    2580 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:41:51.570580 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.565821    2580 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:41:51.570580 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.565825    2580 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:41:51.570580 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.565829    2580 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:41:51.570580 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.565836    2580 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:41:51.570580 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.565842    2580 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:41:51.570580 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.565847    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:41:51.570580 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.565852    2580 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:41:51.570580 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.565856    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:41:51.570580 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.565861    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:41:51.570580 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.565865    2580 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:41:51.570580 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.565869    2580 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:41:51.570580 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.565873    2580 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:41:51.571146 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.565877    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:41:51.571146 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.565881    2580 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:41:51.571146 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.565885    2580 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:41:51.571146 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.565890    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:41:51.571146 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.565894    2580 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:41:51.571146 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.565898    2580 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:41:51.571146 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.565902    2580 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:41:51.571146 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.565906    2580 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:41:51.571146 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.565910    2580 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:41:51.571146 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.565914    2580 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:41:51.571146 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.565919    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:41:51.571146 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.565923    2580 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:41:51.571146 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.565930    2580 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:41:51.571146 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.565934    2580 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:41:51.571146 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.565938    2580 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:41:51.571146 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.565942    2580 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:41:51.571146 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.565946    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:41:51.571146 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.566087    2580 flags.go:64] FLAG: --address="0.0.0.0"
Oct 30 21:41:51.571146 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.566099    2580 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Oct 30 21:41:51.571146 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.566108    2580 flags.go:64] FLAG: --anonymous-auth="true"
Oct 30 21:41:51.571146 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.566115    2580 flags.go:64] FLAG: --application-metrics-count-limit="100"
Oct 30 21:41:51.571699 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.566122    2580 flags.go:64] FLAG: --authentication-token-webhook="false"
Oct 30 21:41:51.571699 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.566127    2580 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Oct 30 21:41:51.571699 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.566134    2580 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Oct 30 21:41:51.571699 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.566141    2580 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Oct 30 21:41:51.571699 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.566146    2580 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Oct 30 21:41:51.571699 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.566151    2580 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Oct 30 21:41:51.571699 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.566157    2580 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Oct 30 21:41:51.571699 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.566162    2580 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Oct 30 21:41:51.571699 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.566167    2580 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Oct 30 21:41:51.571699 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.566172    2580 flags.go:64] FLAG: --cgroup-root=""
Oct 30 21:41:51.571699 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.566177    2580 flags.go:64] FLAG: --cgroups-per-qos="true"
Oct 30 21:41:51.571699 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.566181    2580 flags.go:64] FLAG: --client-ca-file=""
Oct 30 21:41:51.571699 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.566186    2580 flags.go:64] FLAG: --cloud-provider="external"
Oct 30 21:41:51.571699 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.566191    2580 flags.go:64] FLAG: --cluster-dns="[]"
Oct 30 21:41:51.571699 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.566198    2580 flags.go:64] FLAG: --cluster-domain=""
Oct 30 21:41:51.571699 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.566202    2580 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Oct 30 21:41:51.571699 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.566207    2580 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Oct 30 21:41:51.571699 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.566212    2580 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Oct 30 21:41:51.571699 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.566218    2580 flags.go:64] FLAG: --container-log-max-files="5"
Oct 30 21:41:51.571699 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.566224    2580 flags.go:64] FLAG: --container-log-max-size="10Mi"
Oct 30 21:41:51.571699 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.566228    2580 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Oct 30 21:41:51.571699 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.566233    2580 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Oct 30 21:41:51.571699 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.566239    2580 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Oct 30 21:41:51.572256 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.566247    2580 flags.go:64] FLAG: --contention-profiling="false"
Oct 30 21:41:51.572256 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.566251    2580 flags.go:64] FLAG: --cpu-cfs-quota="true"
Oct 30 21:41:51.572256 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.566256    2580 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Oct 30 21:41:51.572256 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.566262    2580 flags.go:64] FLAG: --cpu-manager-policy="none"
Oct 30 21:41:51.572256 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.566266    2580 flags.go:64] FLAG: --cpu-manager-policy-options=""
Oct 30 21:41:51.572256 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.566273    2580 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Oct 30 21:41:51.572256 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.566279    2580 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Oct 30 21:41:51.572256 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.566284    2580 flags.go:64] FLAG: --enable-debugging-handlers="true"
Oct 30 21:41:51.572256 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.566289    2580 flags.go:64] FLAG: --enable-load-reader="false"
Oct 30 21:41:51.572256 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.566294    2580 flags.go:64] FLAG: --enable-server="true"
Oct 30 21:41:51.572256 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.566318    2580 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Oct 30 21:41:51.572256 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.566324    2580 flags.go:64] FLAG: --event-burst="100"
Oct 30 21:41:51.572256 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.566328    2580 flags.go:64] FLAG: --event-qps="50"
Oct 30 21:41:51.572256 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.566333    2580 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Oct 30 21:41:51.572256 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.566338    2580 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Oct 30 21:41:51.572256 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.566343    2580 flags.go:64] FLAG: --eviction-hard=""
Oct 30 21:41:51.572256 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.566350    2580 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Oct 30 21:41:51.572256 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.566354    2580 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Oct 30 21:41:51.572256 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.566359    2580 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Oct 30 21:41:51.572256 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.566365    2580 flags.go:64] FLAG: --eviction-soft=""
Oct 30 21:41:51.572256 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.566369    2580 flags.go:64] FLAG: --eviction-soft-grace-period=""
Oct 30 21:41:51.572256 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.566374    2580 flags.go:64] FLAG: --exit-on-lock-contention="false"
Oct 30 21:41:51.572256 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.566379    2580 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Oct 30 21:41:51.572256 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.566384    2580 flags.go:64] FLAG: --experimental-mounter-path=""
Oct 30 21:41:51.572256 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.566388    2580 flags.go:64] FLAG: --fail-cgroupv1="true"
Oct 30 21:41:51.572903 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.566393    2580 flags.go:64] FLAG: --fail-swap-on="true"
Oct 30 21:41:51.572903 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.566397    2580 flags.go:64] FLAG: --feature-gates=""
Oct 30 21:41:51.572903 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.566404    2580 flags.go:64] FLAG: --file-check-frequency="20s"
Oct 30 21:41:51.572903 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.566409    2580 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Oct 30 21:41:51.572903 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.566414    2580 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Oct 30 21:41:51.572903 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.566419    2580 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Oct 30 21:41:51.572903 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.566424    2580 flags.go:64] FLAG: --healthz-port="10248"
Oct 30 21:41:51.572903 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.566431    2580 flags.go:64] FLAG: --help="false"
Oct 30 21:41:51.572903 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.566436    2580 flags.go:64] FLAG: --hostname-override="ip-10-0-140-18.us-west-1.compute.internal"
Oct 30 21:41:51.572903 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.566442    2580 flags.go:64] FLAG: --housekeeping-interval="10s"
Oct 30 21:41:51.572903 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.566446    2580 flags.go:64] FLAG: --http-check-frequency="20s"
Oct 30 21:41:51.572903 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.566451    2580 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Oct 30 21:41:51.572903 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.566457    2580 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Oct 30 21:41:51.572903 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.566463    2580 flags.go:64] FLAG: --image-gc-high-threshold="85"
Oct 30 21:41:51.572903 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.566468    2580 flags.go:64] FLAG: --image-gc-low-threshold="80"
Oct 30 21:41:51.572903 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.566473    2580 flags.go:64] FLAG: --image-service-endpoint=""
Oct 30 21:41:51.572903 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.566477    2580 flags.go:64] FLAG: --kernel-memcg-notification="false"
Oct 30 21:41:51.572903 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.566482    2580 flags.go:64] FLAG: --kube-api-burst="100"
Oct 30 21:41:51.572903 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.566486    2580 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Oct 30 21:41:51.572903 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.566491    2580 flags.go:64] FLAG: --kube-api-qps="50"
Oct 30 21:41:51.572903 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.566496    2580 flags.go:64] FLAG: --kube-reserved=""
Oct 30 21:41:51.572903 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.566502    2580 flags.go:64] FLAG: --kube-reserved-cgroup=""
Oct 30 21:41:51.572903 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.566507    2580 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Oct 30 21:41:51.572903 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.566512    2580 flags.go:64] FLAG: --kubelet-cgroups=""
Oct 30 21:41:51.573488 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.566516    2580 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Oct 30 21:41:51.573488 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.566521    2580 flags.go:64] FLAG: --lock-file=""
Oct 30 21:41:51.573488 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.566526    2580 flags.go:64] FLAG: --log-cadvisor-usage="false"
Oct 30 21:41:51.573488 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.566530    2580 flags.go:64] FLAG: --log-flush-frequency="5s"
Oct 30 21:41:51.573488 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.566535    2580 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Oct 30 21:41:51.573488 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.566544    2580 flags.go:64] FLAG: --log-json-split-stream="false"
Oct 30 21:41:51.573488 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.566549    2580 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Oct 30 21:41:51.573488 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.566554    2580 flags.go:64] FLAG: --log-text-split-stream="false"
Oct 30 21:41:51.573488 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.566559    2580 flags.go:64] FLAG: --logging-format="text"
Oct 30 21:41:51.573488 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.566563    2580 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Oct 30 21:41:51.573488 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.566568    2580 flags.go:64] FLAG: --make-iptables-util-chains="true"
Oct 30 21:41:51.573488 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.566573    2580 flags.go:64] FLAG: --manifest-url=""
Oct 30 21:41:51.573488 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.566578    2580 flags.go:64] FLAG: --manifest-url-header=""
Oct 30 21:41:51.573488 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.566585    2580 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Oct 30 21:41:51.573488 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.566590    2580 flags.go:64] FLAG: --max-open-files="1000000"
Oct 30 21:41:51.573488 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.566598    2580 flags.go:64] FLAG: --max-pods="110"
Oct 30 21:41:51.573488 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.566603    2580 flags.go:64] FLAG: --maximum-dead-containers="-1"
Oct 30 21:41:51.573488 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.566608    2580 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Oct 30 21:41:51.573488 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.566613    2580 flags.go:64] FLAG: --memory-manager-policy="None"
Oct 30 21:41:51.573488 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.566617    2580 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Oct 30 21:41:51.573488 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.566622    2580 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Oct 30 21:41:51.573488 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.566628    2580 flags.go:64] FLAG: --node-ip="0.0.0.0"
Oct 30 21:41:51.573488 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.566632    2580 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Oct 30 21:41:51.573488 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.566645    2580 flags.go:64] FLAG: --node-status-max-images="50"
Oct 30 21:41:51.573488 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.566649    2580 flags.go:64] FLAG: --node-status-update-frequency="10s"
Oct 30 21:41:51.574101 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.566655    2580 flags.go:64] FLAG: --oom-score-adj="-999"
Oct 30 21:41:51.574101 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.566659    2580 flags.go:64] FLAG: --pod-cidr=""
Oct 30 21:41:51.574101 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.566664    2580 flags.go:64] FLAG: --pod-manifest-path=""
Oct 30 21:41:51.574101 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.566669    2580 flags.go:64] FLAG: --pod-max-pids="-1"
Oct 30 21:41:51.574101 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.566674    2580 flags.go:64] FLAG: --pods-per-core="0"
Oct 30 21:41:51.574101 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.566678    2580 flags.go:64] FLAG: --port="10250"
Oct 30 21:41:51.574101 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.566683    2580 flags.go:64] FLAG: --protect-kernel-defaults="false"
Oct 30 21:41:51.574101 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.566688    2580 flags.go:64] FLAG: --provider-id="aws:///us-west-1a/i-0667f248f24227d44"
Oct 30 21:41:51.574101 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.566693    2580 flags.go:64] FLAG: --qos-reserved=""
Oct 30 21:41:51.574101 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.566698    2580 flags.go:64] FLAG: --read-only-port="10255"
Oct 30 21:41:51.574101 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.566703    2580 flags.go:64] FLAG: --register-node="true"
Oct 30 21:41:51.574101 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.566708    2580 flags.go:64] FLAG: --register-with-taints=""
Oct 30 21:41:51.574101 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.566714    2580 flags.go:64] FLAG: --registry-burst="10"
Oct 30 21:41:51.574101 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.566719    2580 flags.go:64] FLAG: --registry-qps="5"
Oct 30 21:41:51.574101 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.566724    2580 flags.go:64] FLAG: --reserved-cpus=""
Oct 30 21:41:51.574101 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.566728    2580 flags.go:64] FLAG: --reserved-memory=""
Oct 30 21:41:51.574101 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.566734    2580 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Oct 30 21:41:51.574101 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.566739    2580 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Oct 30 21:41:51.574101 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.566743    2580 flags.go:64] FLAG: --rotate-certificates="false"
Oct 30 21:41:51.574101 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.566749    2580 flags.go:64] FLAG: --rotate-server-certificates="false"
Oct 30 21:41:51.574101 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.566753    2580 flags.go:64] FLAG: --runonce="false"
Oct 30 21:41:51.574101 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.566758    2580 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Oct 30 21:41:51.574101 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.566765    2580 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Oct 30 21:41:51.574101 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.566770    2580 flags.go:64] FLAG: --seccomp-default="false"
Oct 30 21:41:51.574101 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.566777    2580 flags.go:64] FLAG: --serialize-image-pulls="true"
Oct 30 21:41:51.574101 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.566782    2580 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Oct 30 21:41:51.574101 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.566787    2580 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Oct 30 21:41:51.574906 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.566792    2580 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Oct 30 21:41:51.574906 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.566799    2580 flags.go:64] FLAG: --storage-driver-password="root"
Oct 30 21:41:51.574906 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.566804    2580 flags.go:64] FLAG: --storage-driver-secure="false"
Oct 30 21:41:51.574906 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.566808    2580 flags.go:64] FLAG: --storage-driver-table="stats"
Oct 30 21:41:51.574906 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.566813    2580 flags.go:64] FLAG: --storage-driver-user="root"
Oct 30 21:41:51.574906 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.566818    2580 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Oct 30 21:41:51.574906 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.566823    2580 flags.go:64] FLAG: --sync-frequency="1m0s"
Oct 30 21:41:51.574906 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.566827    2580 flags.go:64] FLAG: --system-cgroups=""
Oct 30 21:41:51.574906 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.566832    2580 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Oct 30 21:41:51.574906 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.566840    2580 flags.go:64] FLAG: --system-reserved-cgroup=""
Oct 30 21:41:51.574906 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.566845    2580 flags.go:64] FLAG: --tls-cert-file=""
Oct 30 21:41:51.574906 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.566849    2580 flags.go:64] FLAG: --tls-cipher-suites="[]"
Oct 30 21:41:51.574906 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.566857    2580 flags.go:64] FLAG: --tls-min-version=""
Oct 30 21:41:51.574906 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.566861    2580 flags.go:64] FLAG: --tls-private-key-file=""
Oct 30 21:41:51.574906 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.566866    2580 flags.go:64] FLAG: --topology-manager-policy="none"
Oct 30 21:41:51.574906 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.566871    2580 flags.go:64] FLAG: --topology-manager-policy-options=""
Oct 30 21:41:51.574906 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.566876    2580 flags.go:64] FLAG: --topology-manager-scope="container"
Oct 30 21:41:51.574906 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.566881    2580 flags.go:64] FLAG: --v="2"
Oct 30 21:41:51.574906 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.566888    2580 flags.go:64] FLAG: --version="false"
Oct 30 21:41:51.574906 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.566894    2580 flags.go:64] FLAG: --vmodule=""
Oct 30 21:41:51.574906 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.566901    2580 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Oct 30 21:41:51.574906 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.566932    2580 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Oct 30 21:41:51.574906 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.567158    2580 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:41:51.574906 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.567164    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:41:51.575576 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.567170    2580 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:41:51.575576 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.567174    2580 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:41:51.575576 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.567179    2580 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:41:51.575576 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.567186    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:41:51.575576 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.567190    2580 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:41:51.575576 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.567195    2580 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:41:51.575576 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.567200    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:41:51.575576 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.567205    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:41:51.575576 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.567209    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:41:51.575576 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.567215    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:41:51.575576 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.567219    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:41:51.575576 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.567224    2580 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:41:51.575576 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.567228    2580 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:41:51.575576 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.567232    2580 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:41:51.575576 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.567237    2580 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:41:51.575576 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.567241    2580 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:41:51.575576 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.567245    2580 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:41:51.575576 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.567250    2580 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:41:51.575576 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.567255    2580 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:41:51.575576 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.567259    2580 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:41:51.576051 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.567263    2580 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:41:51.576051 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.567267    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:41:51.576051 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.567271    2580 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:41:51.576051 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.567276    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:41:51.576051 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.567280    2580 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:41:51.576051 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.567284    2580 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:41:51.576051 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.567288    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:41:51.576051 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.567292    2580 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:41:51.576051 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.567315    2580 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:41:51.576051 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.567320    2580 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:41:51.576051 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.567325    2580 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:41:51.576051 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.567329    2580 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:41:51.576051 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.567333    2580 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:41:51.576051 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.567337    2580 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:41:51.576051 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.567341    2580 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:41:51.576051 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.567346    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:41:51.576051 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.567351    2580 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:41:51.576051 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.567355    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:41:51.576051 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.567359    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:41:51.576533 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.567364    2580 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:41:51.576533 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.567369    2580 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:41:51.576533 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.567375    2580 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:41:51.576533 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.567379    2580 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:41:51.576533 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.567383    2580 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:41:51.576533 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.567387    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:41:51.576533 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.567391    2580 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:41:51.576533 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.567395    2580 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:41:51.576533 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.567399    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:41:51.576533 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.567403    2580 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:41:51.576533 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.567408    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:41:51.576533 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.567412    2580 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:41:51.576533 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.567416    2580 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:41:51.576533 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.567420    2580 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:41:51.576533 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.567424    2580 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:41:51.576533 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.567429    2580 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:41:51.576533 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.567433    2580 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:41:51.576533 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.567437    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:41:51.576533 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.567442    2580 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:41:51.576533 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.567446    2580 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:41:51.577012 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.567450    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:41:51.577012 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.567455    2580 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:41:51.577012 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.567459    2580 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:41:51.577012 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.567464    2580 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:41:51.577012 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.567468    2580 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:41:51.577012 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.567472    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:41:51.577012 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.567478    2580 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:41:51.577012 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.567484    2580 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:41:51.577012 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.567491    2580 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:41:51.577012 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.567496    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:41:51.577012 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.567500    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:41:51.577012 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.567504    2580 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:41:51.577012 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.567508    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:41:51.577012 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.567514    2580 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:41:51.577012 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.567520    2580 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:41:51.577012 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.567526    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:41:51.577012 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.567530    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:41:51.577012 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.567534    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:41:51.577012 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.567538    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:41:51.577012 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.567543    2580 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:41:51.577519 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.567547    2580 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:41:51.577519 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.567551    2580 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:41:51.577519 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.567556    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:41:51.577519 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.567560    2580 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:41:51.577519 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.567564    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:41:51.577519 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.567568    2580 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:41:51.577519 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.567573    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:41:51.577519 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.567577    2580 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:41:51.577519 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.567581    2580 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:41:51.577519 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.567585    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:41:51.577519 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.567590    2580 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:41:51.577519 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.567594    2580 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:41:51.577519 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.567598    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:41:51.577519 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.567602    2580 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:41:51.577519 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.567606    2580 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:41:51.577519 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.567610    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:41:51.577519 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.567615    2580 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:41:51.577519 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.567619    2580 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:41:51.577519 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.567623    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:41:51.577519 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.567627    2580 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:41:51.578004 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.567632    2580 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:41:51.578004 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.567637    2580 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:41:51.578004 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.567641    2580 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:41:51.578004 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.567647    2580 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:41:51.578004 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.567653    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:41:51.578004 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.567700    2580 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 21:41:51.578004 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.567771    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:41:51.578004 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.567783    2580 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:41:51.578004 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.567788    2580 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:41:51.578004 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.567792    2580 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:41:51.578004 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.567797    2580 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:41:51.578004 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.567801    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:41:51.578004 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.567808    2580 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:41:51.578004 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.567814    2580 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:41:51.578004 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.567820    2580 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:41:51.578004 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.567824    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:41:51.578404 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.567828    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:41:51.578404 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.567833    2580 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:41:51.578404 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.567837    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:41:51.578404 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.567841    2580 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:41:51.578404 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.567846    2580 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:41:51.578404 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.567851    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:41:51.578404 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.567855    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:41:51.578404 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.567859    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:41:51.578404 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.567864    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:41:51.578404 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.567868    2580 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:41:51.578404 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.567872    2580 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:41:51.578404 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.567876    2580 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:41:51.578404 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.567881    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:41:51.578404 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.567885    2580 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:41:51.578404 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.567889    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:41:51.578404 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.567893    2580 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:41:51.578404 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.567897    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:41:51.578404 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.567902    2580 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:41:51.578404 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.567906    2580 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:41:51.578404 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.567910    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:41:51.578881 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.567914    2580 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:41:51.578881 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.567919    2580 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:41:51.578881 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.567924    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:41:51.578881 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.567929    2580 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:41:51.578881 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.567934    2580 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:41:51.578881 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.567938    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:41:51.578881 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.567942    2580 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:41:51.578881 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.567946    2580 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:41:51.578881 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.567950    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:41:51.578881 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.567954    2580 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:41:51.578881 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.567958    2580 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:41:51.578881 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.567962    2580 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:41:51.578881 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.567966    2580 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:41:51.578881 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.567973    2580 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:41:51.578881 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.567978    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:41:51.578881 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.567983    2580 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:41:51.578881 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.567987    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:41:51.578881 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.567993    2580 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:41:51.578881 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.567997    2580 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:41:51.579365 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.568001    2580 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:41:51.579365 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.568005    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:41:51.579365 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.568009    2580 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:41:51.579365 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.568013    2580 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:41:51.579365 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.568018    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:41:51.579365 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.568022    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:41:51.579365 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.568026    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:41:51.579365 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.568030    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:41:51.579365 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.568034    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:41:51.579365 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.568039    2580 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:41:51.579365 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.568043    2580 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:41:51.579365 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.568048    2580 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:41:51.579365 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.568052    2580 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:41:51.579365 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.568057    2580 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:41:51.579365 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.568061    2580 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:41:51.579365 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.568067    2580 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:41:51.579365 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.568071    2580 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:41:51.579365 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.568076    2580 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:41:51.579365 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.568080    2580 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:41:51.579825 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.568084    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:41:51.579825 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.568089    2580 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:41:51.579825 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.568093    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:41:51.579825 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.568097    2580 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:41:51.579825 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.568101    2580 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:41:51.579825 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.568105    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:41:51.579825 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.568109    2580 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:41:51.579825 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.568113    2580 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:41:51.579825 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.568118    2580 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:41:51.579825 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.568122    2580 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:41:51.579825 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.568126    2580 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:41:51.579825 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.568130    2580 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:41:51.579825 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.568135    2580 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:41:51.579825 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.568139    2580 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:41:51.579825 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.568143    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:41:51.579825 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.568147    2580 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:41:51.579825 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.568151    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:41:51.579825 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.568155    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:41:51.579825 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.568160    2580 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:41:51.580278 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.568164    2580 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:41:51.580278 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.568168    2580 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:41:51.580278 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.568172    2580 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:41:51.580278 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.568176    2580 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:41:51.580278 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.568180    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:41:51.580278 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.568185    2580 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:41:51.580278 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.568189    2580 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:41:51.580278 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.568193    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:41:51.580278 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.568197    2580 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:41:51.580278 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.568203    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:41:51.580278 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.568207    2580 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:41:51.580278 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.568212    2580 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:41:51.580278 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.568217    2580 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:41:51.580278 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.568221    2580 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:41:51.580278 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.568239    2580 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:41:51.580278 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.568247    2580 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:41:51.580278 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.568251    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:41:51.580278 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.568255    2580 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:41:51.580278 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.568259    2580 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:41:51.580278 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.575837    2580 server.go:525] "Kubelet version" kubeletVersion="v1.35.9"
Oct 30 21:41:51.580776 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.575857    2580 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Oct 30 21:41:51.580776 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.575872    2580 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Oct 30 21:41:51.580776 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.575877    2580 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Oct 30 21:41:51.580776 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.575939    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:41:51.580776 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.575945    2580 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:41:51.580776 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.575949    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:41:51.580776 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.575951    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:41:51.580776 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.575954    2580 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:41:51.580776 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.575957    2580 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:41:51.580776 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.575959    2580 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:41:51.580776 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.575962    2580 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:41:51.580776 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.575965    2580 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:41:51.580776 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.575967    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:41:51.580776 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.575970    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:41:51.580776 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.575973    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:41:51.580776 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.575976    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:41:51.580776 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.575978    2580 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:41:51.580776 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.575981    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:41:51.580776 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.575984    2580 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:41:51.581262 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.575986    2580 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:41:51.581262 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.575989    2580 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:41:51.581262 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.575991    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:41:51.581262 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.575994    2580 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:41:51.581262 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.575996    2580 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:41:51.581262 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.575999    2580 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:41:51.581262 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576002    2580 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:41:51.581262 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576005    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:41:51.581262 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576007    2580 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:41:51.581262 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576010    2580 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:41:51.581262 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576013    2580 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:41:51.581262 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576015    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:41:51.581262 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576018    2580 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:41:51.581262 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576022    2580 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:41:51.581262 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576024    2580 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:41:51.581262 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576029    2580 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:41:51.581262 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576034    2580 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:41:51.581262 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576037    2580 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:41:51.581262 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576040    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:41:51.581770 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576043    2580 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:41:51.581770 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576046    2580 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:41:51.581770 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576049    2580 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:41:51.581770 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576051    2580 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:41:51.581770 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576054    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:41:51.581770 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576057    2580 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:41:51.581770 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576060    2580 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:41:51.581770 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576063    2580 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:41:51.581770 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576066    2580 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:41:51.581770 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576068    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:41:51.581770 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576071    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:41:51.581770 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576074    2580 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:41:51.581770 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576077    2580 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:41:51.581770 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576080    2580 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:41:51.581770 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576082    2580 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:41:51.581770 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576085    2580 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:41:51.581770 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576088    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:41:51.581770 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576090    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:41:51.581770 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576093    2580 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:41:51.582218 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576096    2580 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:41:51.582218 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576099    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:41:51.582218 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576101    2580 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:41:51.582218 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576104    2580 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:41:51.582218 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576107    2580 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:41:51.582218 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576109    2580 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:41:51.582218 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576112    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:41:51.582218 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576115    2580 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:41:51.582218 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576118    2580 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:41:51.582218 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576122    2580 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:41:51.582218 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576124    2580 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:41:51.582218 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576127    2580 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:41:51.582218 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576130    2580 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:41:51.582218 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576133    2580 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:41:51.582218 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576135    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:41:51.582218 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576138    2580 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:41:51.582218 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576140    2580 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:41:51.582218 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576144    2580 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:41:51.582218 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576147    2580 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:41:51.582218 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576150    2580 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:41:51.582910 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576153    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:41:51.582910 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576156    2580 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:41:51.582910 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576158    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:41:51.582910 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576161    2580 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:41:51.582910 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576163    2580 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:41:51.582910 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576166    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:41:51.582910 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576169    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:41:51.582910 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576171    2580 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:41:51.582910 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576174    2580 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:41:51.582910 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576176    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:41:51.582910 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576179    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:41:51.582910 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576181    2580 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:41:51.582910 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576184    2580 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:41:51.582910 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576187    2580 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:41:51.582910 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576189    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:41:51.582910 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576192    2580 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:41:51.582910 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576195    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:41:51.582910 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576197    2580 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:41:51.582910 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576200    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:41:51.583612 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576203    2580 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:41:51.583612 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576205    2580 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:41:51.583612 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576208    2580 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:41:51.583612 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576211    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:41:51.583612 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576214    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:41:51.583612 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576216    2580 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:41:51.583612 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576219    2580 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:41:51.583612 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576221    2580 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:41:51.583612 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576224    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:41:51.583612 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576234    2580 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:41:51.583612 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576237    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:41:51.583612 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576240    2580 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:41:51.583612 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576242    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:41:51.583612 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.576272    2580 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 21:41:51.583612 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576436    2580 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:41:51.583612 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576442    2580 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:41:51.584090 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576446    2580 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:41:51.584090 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576448    2580 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:41:51.584090 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576451    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:41:51.584090 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576454    2580 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:41:51.584090 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576456    2580 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:41:51.584090 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576459    2580 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:41:51.584090 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576461    2580 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:41:51.584090 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576464    2580 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:41:51.584090 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576466    2580 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:41:51.584090 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576469    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:41:51.584090 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576471    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:41:51.584090 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576474    2580 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:41:51.584090 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576476    2580 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:41:51.584090 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576478    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:41:51.584090 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576481    2580 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:41:51.584090 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576483    2580 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:41:51.584090 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576488    2580 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:41:51.584090 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576492    2580 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:41:51.584090 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576495    2580 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:41:51.584791 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576499    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:41:51.584791 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576501    2580 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:41:51.584791 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576504    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:41:51.584791 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576506    2580 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:41:51.584791 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576509    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:41:51.584791 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576511    2580 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:41:51.584791 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576514    2580 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:41:51.584791 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576517    2580 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:41:51.584791 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576519    2580 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:41:51.584791 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576521    2580 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:41:51.584791 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576524    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:41:51.584791 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576526    2580 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:41:51.584791 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576529    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:41:51.584791 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576531    2580 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:41:51.584791 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576534    2580 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:41:51.584791 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576536    2580 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:41:51.584791 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576539    2580 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:41:51.584791 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576542    2580 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:41:51.584791 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576544    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:41:51.584791 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576547    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:41:51.585440 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576550    2580 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:41:51.585440 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576552    2580 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:41:51.585440 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576554    2580 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:41:51.585440 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576557    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:41:51.585440 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576559    2580 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:41:51.585440 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576562    2580 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:41:51.585440 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576566    2580 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:41:51.585440 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576569    2580 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:41:51.585440 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576572    2580 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:41:51.585440 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576575    2580 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:41:51.585440 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576578    2580 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:41:51.585440 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576580    2580 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:41:51.585440 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576583    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:41:51.585440 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576586    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:41:51.585440 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576588    2580 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:41:51.585440 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576591    2580 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:41:51.585440 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576593    2580 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:41:51.585440 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576595    2580 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:41:51.585440 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576598    2580 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:41:51.585920 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576601    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:41:51.585920 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576603    2580 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:41:51.585920 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576606    2580 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:41:51.585920 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576609    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:41:51.585920 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576611    2580 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:41:51.585920 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576614    2580 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:41:51.585920 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576616    2580 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:41:51.585920 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576619    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:41:51.585920 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576621    2580 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:41:51.585920 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576624    2580 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:41:51.585920 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576626    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:41:51.585920 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576629    2580 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:41:51.585920 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576632    2580 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:41:51.585920 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576634    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:41:51.585920 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576637    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:41:51.585920 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576639    2580 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:41:51.585920 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576642    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:41:51.585920 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576645    2580 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:41:51.585920 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576647    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:41:51.585920 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576650    2580 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:41:51.586547 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576652    2580 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:41:51.586547 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576655    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:41:51.586547 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576657    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:41:51.586547 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576660    2580 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:41:51.586547 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576662    2580 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:41:51.586547 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576665    2580 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:41:51.586547 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576667    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:41:51.586547 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576671    2580 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:41:51.586547 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576673    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:41:51.586547 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576676    2580 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:41:51.586547 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576678    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:41:51.586547 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576680    2580 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:41:51.586547 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576683    2580 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:41:51.586547 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576685    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:41:51.586547 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576688    2580 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:41:51.586547 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576690    2580 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:41:51.586547 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576693    2580 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:41:51.586547 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576695    2580 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:41:51.586547 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576698    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:41:51.586547 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576701    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:41:51.587018 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576703    2580 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:41:51.587018 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576705    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:41:51.587018 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576708    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:41:51.587018 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576710    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:41:51.587018 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576713    2580 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:41:51.587018 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576715    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:41:51.587018 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.576739    2580 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 21:41:51.587018 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576776    2580 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:41:51.587018 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576780    2580 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:41:51.587018 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576783    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:41:51.587018 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576786    2580 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:41:51.587018 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576788    2580 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:41:51.587018 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576793    2580 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:41:51.587018 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576797    2580 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:41:51.587018 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576800    2580 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:41:51.587386 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576803    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:41:51.587386 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576806    2580 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:41:51.587386 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576808    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:41:51.587386 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576811    2580 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:41:51.587386 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576813    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:41:51.587386 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576817    2580 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:41:51.587386 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576819    2580 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:41:51.587386 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576822    2580 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:41:51.587386 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576824    2580 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:41:51.587386 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576827    2580 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:41:51.587386 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576830    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:41:51.587386 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576832    2580 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:41:51.587386 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576835    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:41:51.587386 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576837    2580 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:41:51.587386 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576840    2580 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:41:51.587386 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576842    2580 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:41:51.587386 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576845    2580 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:41:51.587386 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576848    2580 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:41:51.587386 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576850    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:41:51.587386 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576853    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:41:51.587842 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576855    2580 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:41:51.587842 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576857    2580 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:41:51.587842 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576860    2580 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:41:51.587842 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576863    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:41:51.587842 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576865    2580 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:41:51.587842 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576868    2580 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:41:51.587842 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576871    2580 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:41:51.587842 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576875    2580 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:41:51.587842 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576878    2580 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:41:51.587842 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576881    2580 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:41:51.587842 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576886    2580 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:41:51.587842 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576888    2580 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:41:51.587842 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576891    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:41:51.587842 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576893    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:41:51.587842 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576896    2580 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:41:51.587842 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576898    2580 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:41:51.587842 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576901    2580 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:41:51.587842 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576903    2580 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:41:51.587842 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576906    2580 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:41:51.588316 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576909    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:41:51.588316 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576913    2580 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:41:51.588316 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576915    2580 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:41:51.588316 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576919    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:41:51.588316 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576921    2580 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:41:51.588316 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576924    2580 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:41:51.588316 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576927    2580 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:41:51.588316 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576929    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:41:51.588316 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576932    2580 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:41:51.588316 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576935    2580 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:41:51.588316 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576937    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:41:51.588316 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576940    2580 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:41:51.588316 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576942    2580 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:41:51.588316 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576945    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:41:51.588316 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576948    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:41:51.588316 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576950    2580 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:41:51.588316 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576953    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:41:51.588316 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576955    2580 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:41:51.588316 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576958    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:41:51.588316 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576960    2580 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:41:51.588860 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576963    2580 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:41:51.588860 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576965    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:41:51.588860 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576968    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:41:51.588860 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576972    2580 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:41:51.588860 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576974    2580 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:41:51.588860 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576977    2580 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:41:51.588860 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576979    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:41:51.588860 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576982    2580 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:41:51.588860 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576985    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:41:51.588860 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576987    2580 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:41:51.588860 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576990    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:41:51.588860 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576992    2580 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:41:51.588860 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576995    2580 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:41:51.588860 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.576999    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:41:51.588860 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.577001    2580 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:41:51.588860 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.577004    2580 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:41:51.588860 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.577007    2580 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:41:51.588860 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.577009    2580 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:41:51.588860 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.577012    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:41:51.588860 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.577014    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:41:51.589350 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.577017    2580 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:41:51.589350 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.577020    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:41:51.589350 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.577022    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:41:51.589350 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.577025    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:41:51.589350 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.577028    2580 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:41:51.589350 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.577030    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:41:51.589350 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.577033    2580 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:41:51.589350 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.577035    2580 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:41:51.589350 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.577038    2580 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:41:51.589350 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.577040    2580 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:41:51.589350 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.577043    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:41:51.589350 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.577045    2580 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:41:51.589350 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.577047    2580 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:41:51.589350 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.577050    2580 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:41:51.589350 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.577053    2580 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:41:51.589350 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.577056    2580 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:41:51.589350 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.577058    2580 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:41:51.589350 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.577062    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:41:51.589350 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:51.577064    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:41:51.589350 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.577126    2580 server.go:957] "Client rotation is on, will bootstrap in background"
Oct 30 21:41:51.589823 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.579480    2580 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Oct 30 21:41:51.589823 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.580494    2580 server.go:1014] "Starting client certificate rotation"
Oct 30 21:41:51.589823 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.580592    2580 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Oct 30 21:41:51.589823 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.580640    2580 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Oct 30 21:41:51.589823 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.584510    2580 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 30 21:41:51.591218 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.591197    2580 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 30 21:41:51.601286 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.601261    2580 log.go:25] "Validated CRI v1 runtime API"
Oct 30 21:41:51.605131 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.605114    2580 log.go:25] "Validated CRI v1 image API"
Oct 30 21:41:51.606003 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.605974    2580 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Oct 30 21:41:51.611365 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.611342    2580 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 f6aaa1c3-3c60-4fad-b668-80a24b2419cb:/dev/nvme0n1p4 f6e04ef7-2419-486f-af3a-219a87e81add:/dev/nvme0n1p3]
Oct 30 21:41:51.611432 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.611361    2580 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0}]
Oct 30 21:41:51.616761 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.616638    2580 manager.go:217] Machine: {Timestamp:2026-10-30 21:41:51.615106418 +0000 UTC m=+0.306669166 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3095778 MemoryCapacity:32812015616 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2f47f72d295f8172bcce343315896e SystemUUID:ec2f47f7-2d29-5f81-72bc-ce343315896e BootID:ee1b8de9-2692-4004-9fb2-0c9e0790d8fd Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16406007808 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16406007808 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6562406400 Type:vfs Inodes:819200 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:5d:e4:f6:29:1f Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:5d:e4:f6:29:1f Speed:0 Mtu:9001} {Name:ovs-system MacAddress:fa:08:63:32:2d:6a Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812015616 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Oct 30 21:41:51.616761 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.616761    2580 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Oct 30 21:41:51.616885 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.616849    2580 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Oct 30 21:41:51.620344 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.620313    2580 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Oct 30 21:41:51.620494 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.620347    2580 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-140-18.us-west-1.compute.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Oct 30 21:41:51.620536 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.620498    2580 topology_manager.go:143] "Creating topology manager with none policy"
Oct 30 21:41:51.620536 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.620507    2580 container_manager_linux.go:311] "Creating device plugin manager"
Oct 30 21:41:51.620588 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.620548    2580 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 30 21:41:51.620588 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.620566    2580 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 30 21:41:51.621791 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.621779    2580 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager"
Oct 30 21:41:51.626220 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.626207    2580 state_mem.go:41] "Initialized" logger="CPUManager state memory"
Oct 30 21:41:51.626373 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.626364    2580 server.go:1235] "Using root directory" path="/var/lib/kubelet"
Oct 30 21:41:51.627188 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.627178    2580 kubelet.go:493] "Attempting to sync node with API server"
Oct 30 21:41:51.627219 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.627193    2580 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests"
Oct 30 21:41:51.627219 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.627204    2580 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Oct 30 21:41:51.627219 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.627213    2580 kubelet.go:405] "Adding apiserver pod source"
Oct 30 21:41:51.627326 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.627222    2580 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Oct 30 21:41:51.627754 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.627740    2580 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 30 21:41:51.630130 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.630111    2580 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1"
Oct 30 21:41:51.637537 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.637523    2580 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 30 21:41:51.638456 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.638443    2580 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Oct 30 21:41:51.638497 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.638475    2580 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Oct 30 21:41:51.638638 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.638627    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Oct 30 21:41:51.638671 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.638643    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Oct 30 21:41:51.638671 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.638650    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Oct 30 21:41:51.638671 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.638656    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Oct 30 21:41:51.638671 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.638661    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Oct 30 21:41:51.638671 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.638667    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Oct 30 21:41:51.639334 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.639326    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Oct 30 21:41:51.639365 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.639335    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Oct 30 21:41:51.639365 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.639342    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Oct 30 21:41:51.639365 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.639348    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Oct 30 21:41:51.639365 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.639357    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Oct 30 21:41:51.639365 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.639365    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Oct 30 21:41:51.639994 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.639986    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Oct 30 21:41:51.640026 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.639995    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Oct 30 21:41:51.645488 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.645468    2580 server.go:1263] "Started kubelet"
Oct 30 21:41:51.645588 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.645560    2580 server.go:182] "Starting to listen" address="0.0.0.0" port=10250
Oct 30 21:41:51.645633 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.645563    2580 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Oct 30 21:41:51.645671 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.645641    2580 server_v1.go:49] "podresources" method="list" useActivePods=true
Oct 30 21:41:51.646234 ip-10-0-140-18 systemd[1]: Started Kubernetes Kubelet.
Oct 30 21:41:51.647452 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.647438    2580 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Oct 30 21:41:51.648346 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.648331    2580 server.go:317] "Adding debug handlers to kubelet server"
Oct 30 21:41:51.654273 ip-10-0-140-18 kubenswrapper[2580]: E1030 21:41:51.654247    2580 kubelet.go:1679] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Oct 30 21:41:51.655544 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.655527    2580 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer"
Oct 30 21:41:51.655613 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.655525    2580 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Oct 30 21:41:51.655661 ip-10-0-140-18 kubenswrapper[2580]: E1030 21:41:51.655647    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-140-18.us-west-1.compute.internal\" not found"
Oct 30 21:41:51.657328 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.656261    2580 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Oct 30 21:41:51.657328 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.656262    2580 volume_manager.go:309] "The desired_state_of_world populator starts"
Oct 30 21:41:51.657328 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.656290    2580 volume_manager.go:311] "Starting Kubelet Volume Manager"
Oct 30 21:41:51.657328 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.656389    2580 reconstruct.go:97] "Volume reconstruction finished"
Oct 30 21:41:51.657328 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.656399    2580 reconciler.go:29] "Reconciler: start to sync state"
Oct 30 21:41:51.657732 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.657717    2580 factory.go:153] Registering CRI-O factory
Oct 30 21:41:51.657732 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.657734    2580 factory.go:223] Registration of the crio container factory successfully
Oct 30 21:41:51.657838 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.657795    2580 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Oct 30 21:41:51.657838 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.657804    2580 factory.go:55] Registering systemd factory
Oct 30 21:41:51.657838 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.657811    2580 factory.go:223] Registration of the systemd container factory successfully
Oct 30 21:41:51.657838 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.657830    2580 factory.go:103] Registering Raw factory
Oct 30 21:41:51.657954 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.657842    2580 manager.go:1199] Started watching for new ooms in manager
Oct 30 21:41:51.658966 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.658955    2580 manager.go:319] Starting recovery of all containers
Oct 30 21:41:51.668990 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.668968    2580 manager.go:324] Recovery completed
Oct 30 21:41:51.675196 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.675180    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 21:41:51.676991 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.676975    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-18.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Oct 30 21:41:51.677062 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.677010    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-18.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 21:41:51.677062 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.677025    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-18.us-west-1.compute.internal" event="NodeHasSufficientPID"
Oct 30 21:41:51.677550 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.677537    2580 cpu_manager.go:226] "Starting" policy="none"
Oct 30 21:41:51.677623 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.677550    2580 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s"
Oct 30 21:41:51.677623 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.677538    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 21:41:51.677623 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.677572    2580 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory"
Oct 30 21:41:51.678236 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.678219    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-18.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Oct 30 21:41:51.678334 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.678251    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-18.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 21:41:51.678334 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.678265    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-18.us-west-1.compute.internal" event="NodeHasSufficientPID"
Oct 30 21:41:51.679845 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.679832    2580 policy_none.go:50] "Start"
Oct 30 21:41:51.679912 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.679849    2580 memory_manager.go:187] "Starting memorymanager" policy="None"
Oct 30 21:41:51.679912 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.679870    2580 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Oct 30 21:41:51.681945 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.681931    2580 policy_none.go:44] "Start"
Oct 30 21:41:51.721969 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.721935    2580 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"}
Oct 30 21:41:51.724312 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.724285    2580 manager.go:341] "Starting Device Plugin manager"
Oct 30 21:41:51.724372 ip-10-0-140-18 kubenswrapper[2580]: E1030 21:41:51.724345    2580 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Oct 30 21:41:51.724372 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.724359    2580 server.go:87] "Starting device plugin registration server"
Oct 30 21:41:51.724677 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.724663    2580 eviction_manager.go:194] "Eviction manager: starting control loop"
Oct 30 21:41:51.724730 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.724683    2580 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Oct 30 21:41:51.724899 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.724880    2580 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Oct 30 21:41:51.725002 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.724992    2580 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts"
Oct 30 21:41:51.725064 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.725003    2580 plugin_manager.go:121] "Starting Kubelet Plugin Manager"
Oct 30 21:41:51.725346 ip-10-0-140-18 kubenswrapper[2580]: E1030 21:41:51.725326    2580 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Oct 30 21:41:51.725423 ip-10-0-140-18 kubenswrapper[2580]: E1030 21:41:51.725381    2580 eviction_manager.go:297] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-140-18.us-west-1.compute.internal\" not found"
Oct 30 21:41:51.784969 ip-10-0-140-18 kubenswrapper[2580]: E1030 21:41:51.784940    2580 controller.go:201] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-140-18.us-west-1.compute.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Oct 30 21:41:51.784969 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.784945    2580 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-140-18.us-west-1.compute.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Oct 30 21:41:51.786909 ip-10-0-140-18 kubenswrapper[2580]: E1030 21:41:51.784946    2580 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-18.us-west-1.compute.internal.18e36ca0220d106c  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-18.us-west-1.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-140-18.us-west-1.compute.internal,},FirstTimestamp:2026-10-30 21:41:51.64543806 +0000 UTC m=+0.337000789,LastTimestamp:2026-10-30 21:41:51.64543806 +0000 UTC m=+0.337000789,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-18.us-west-1.compute.internal,}"
Oct 30 21:41:51.795868 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.795701    2580 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Oct 30 21:41:51.797188 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.797163    2580 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Oct 30 21:41:51.797263 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.797194    2580 status_manager.go:249] "Starting to sync pod status with apiserver"
Oct 30 21:41:51.797263 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.797218    2580 kubelet.go:2521] "Starting kubelet main sync loop"
Oct 30 21:41:51.797366 ip-10-0-140-18 kubenswrapper[2580]: E1030 21:41:51.797259    2580 kubelet.go:2545] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Oct 30 21:41:51.825919 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.825837    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 21:41:51.826780 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.826760    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-18.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Oct 30 21:41:51.826886 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.826795    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-18.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 21:41:51.826886 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.826808    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-18.us-west-1.compute.internal" event="NodeHasSufficientPID"
Oct 30 21:41:51.826886 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.826834    2580 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-140-18.us-west-1.compute.internal"
Oct 30 21:41:51.897942 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.897906    2580 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-18.us-west-1.compute.internal","kube-system/kube-apiserver-proxy-ip-10-0-140-18.us-west-1.compute.internal"]
Oct 30 21:41:51.898098 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.897978    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 21:41:51.898918 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.898903    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-18.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Oct 30 21:41:51.898982 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.898932    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-18.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 21:41:51.898982 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.898943    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-18.us-west-1.compute.internal" event="NodeHasSufficientPID"
Oct 30 21:41:51.901267 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.901255    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 21:41:51.901427 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.901409    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-18.us-west-1.compute.internal"
Oct 30 21:41:51.901480 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.901444    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 21:41:51.902496 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.902481    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-18.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Oct 30 21:41:51.902577 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.902496    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-18.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Oct 30 21:41:51.902577 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.902508    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-18.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 21:41:51.902577 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.902514    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-18.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 21:41:51.902577 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.902523    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-18.us-west-1.compute.internal" event="NodeHasSufficientPID"
Oct 30 21:41:51.902753 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.902524    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-18.us-west-1.compute.internal" event="NodeHasSufficientPID"
Oct 30 21:41:51.904716 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.904700    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-18.us-west-1.compute.internal"
Oct 30 21:41:51.904782 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.904726    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 21:41:51.905503 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.905484    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-18.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Oct 30 21:41:51.905569 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.905521    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-18.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 21:41:51.905569 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.905535    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-18.us-west-1.compute.internal" event="NodeHasSufficientPID"
Oct 30 21:41:51.908887 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.908861    2580 reflector.go:446] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Oct 30 21:41:51.908961 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.908914    2580 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-p528f"
Oct 30 21:41:51.932213 ip-10-0-140-18 kubenswrapper[2580]: E1030 21:41:51.932191    2580 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-18.us-west-1.compute.internal\" not found" node="ip-10-0-140-18.us-west-1.compute.internal"
Oct 30 21:41:51.935117 ip-10-0-140-18 kubenswrapper[2580]: E1030 21:41:51.935100    2580 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-18.us-west-1.compute.internal\" not found" node="ip-10-0-140-18.us-west-1.compute.internal"
Oct 30 21:41:51.943263 ip-10-0-140-18 kubenswrapper[2580]: E1030 21:41:51.943180    2580 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-18.us-west-1.compute.internal.18e36ca023ee85f1  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-18.us-west-1.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-140-18.us-west-1.compute.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-140-18.us-west-1.compute.internal,},FirstTimestamp:2026-10-30 21:41:51.676990961 +0000 UTC m=+0.368553699,LastTimestamp:2026-10-30 21:41:51.676990961 +0000 UTC m=+0.368553699,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-18.us-west-1.compute.internal,}"
Oct 30 21:41:51.943703 ip-10-0-140-18 kubenswrapper[2580]: E1030 21:41:51.943676    2580 reflector.go:204] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.CSIDriver"
Oct 30 21:41:51.959116 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.959091    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/9a45018bbca93ea1080adf030d8fbb7f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-18.us-west-1.compute.internal\" (UID: \"9a45018bbca93ea1080adf030d8fbb7f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-18.us-west-1.compute.internal"
Oct 30 21:41:51.959116 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.959120    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9a45018bbca93ea1080adf030d8fbb7f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-18.us-west-1.compute.internal\" (UID: \"9a45018bbca93ea1080adf030d8fbb7f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-18.us-west-1.compute.internal"
Oct 30 21:41:51.959275 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:51.959135    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/aae6ff4840f4ffd5842f280d0afa74f6-config\") pod \"kube-apiserver-proxy-ip-10-0-140-18.us-west-1.compute.internal\" (UID: \"aae6ff4840f4ffd5842f280d0afa74f6\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-18.us-west-1.compute.internal"
Oct 30 21:41:51.993014 ip-10-0-140-18 kubenswrapper[2580]: E1030 21:41:51.992984    2580 reflector.go:204] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-140-18.us-west-1.compute.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Node"
Oct 30 21:41:51.993014 ip-10-0-140-18 kubenswrapper[2580]: E1030 21:41:51.992988    2580 reflector.go:204] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Service"
Oct 30 21:41:52.008508 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:52.008486    2580 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 21:41:52.010039 ip-10-0-140-18 kubenswrapper[2580]: E1030 21:41:52.010019    2580 kubelet_node_status.go:115] "Unable to register node with API server, error getting existing node" err="nodes \"ip-10-0-140-18.us-west-1.compute.internal\" not found" node="ip-10-0-140-18.us-west-1.compute.internal"
Oct 30 21:41:52.060469 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:52.060438    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9a45018bbca93ea1080adf030d8fbb7f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-18.us-west-1.compute.internal\" (UID: \"9a45018bbca93ea1080adf030d8fbb7f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-18.us-west-1.compute.internal"
Oct 30 21:41:52.060469 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:52.060479    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/aae6ff4840f4ffd5842f280d0afa74f6-config\") pod \"kube-apiserver-proxy-ip-10-0-140-18.us-west-1.compute.internal\" (UID: \"aae6ff4840f4ffd5842f280d0afa74f6\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-18.us-west-1.compute.internal"
Oct 30 21:41:52.060734 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:52.060503    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/9a45018bbca93ea1080adf030d8fbb7f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-18.us-west-1.compute.internal\" (UID: \"9a45018bbca93ea1080adf030d8fbb7f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-18.us-west-1.compute.internal"
Oct 30 21:41:52.060734 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:52.060538    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/9a45018bbca93ea1080adf030d8fbb7f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-18.us-west-1.compute.internal\" (UID: \"9a45018bbca93ea1080adf030d8fbb7f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-18.us-west-1.compute.internal"
Oct 30 21:41:52.060734 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:52.060539    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9a45018bbca93ea1080adf030d8fbb7f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-18.us-west-1.compute.internal\" (UID: \"9a45018bbca93ea1080adf030d8fbb7f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-18.us-west-1.compute.internal"
Oct 30 21:41:52.060734 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:52.060539    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/aae6ff4840f4ffd5842f280d0afa74f6-config\") pod \"kube-apiserver-proxy-ip-10-0-140-18.us-west-1.compute.internal\" (UID: \"aae6ff4840f4ffd5842f280d0afa74f6\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-18.us-west-1.compute.internal"
Oct 30 21:41:52.091110 ip-10-0-140-18 kubenswrapper[2580]: E1030 21:41:52.091019    2580 nodelease.go:50] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-140-18.us-west-1.compute.internal\" not found" node="ip-10-0-140-18.us-west-1.compute.internal"
Oct 30 21:41:52.210401 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:52.210367    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 21:41:52.211401 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:52.211382    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-18.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Oct 30 21:41:52.211501 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:52.211415    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-18.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 21:41:52.211501 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:52.211429    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-18.us-west-1.compute.internal" event="NodeHasSufficientPID"
Oct 30 21:41:52.211501 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:52.211463    2580 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-140-18.us-west-1.compute.internal"
Oct 30 21:41:52.235594 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:52.235566    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-18.us-west-1.compute.internal"
Oct 30 21:41:52.237717 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:52.237699    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-18.us-west-1.compute.internal"
Oct 30 21:41:52.271263 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:52.271236    2580 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-140-18.us-west-1.compute.internal"
Oct 30 21:41:52.271263 ip-10-0-140-18 kubenswrapper[2580]: E1030 21:41:52.271264    2580 kubelet_node_status.go:524] "Error updating node status, will retry" err="error getting node \"ip-10-0-140-18.us-west-1.compute.internal\": node \"ip-10-0-140-18.us-west-1.compute.internal\" not found"
Oct 30 21:41:52.391755 ip-10-0-140-18 kubenswrapper[2580]: E1030 21:41:52.391674    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-140-18.us-west-1.compute.internal\" not found"
Oct 30 21:41:52.492722 ip-10-0-140-18 kubenswrapper[2580]: E1030 21:41:52.492688    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-140-18.us-west-1.compute.internal\" not found"
Oct 30 21:41:52.580007 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:52.579978    2580 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Oct 30 21:41:52.580545 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:52.580136    2580 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:161: Unexpected watch close - watch lasted less than a second and no items received"
Oct 30 21:41:52.580545 ip-10-0-140-18 kubenswrapper[2580]: E1030 21:41:52.580197    2580 event.go:368] "Unable to write event (may retry after sleeping)" err="Patch \"https://ad8669e46bc8f4e2bbe5ac45b704f8a7-5cb0796d801b0643.elb.us-east-2.amazonaws.com:6443/api/v1/namespaces/default/events/ip-10-0-140-18.us-west-1.compute.internal.18e36ca023ef1ab9\": read tcp 10.0.140.18:60286->13.184.211.235:6443: use of closed network connection" event="&Event{ObjectMeta:{ip-10-0-140-18.us-west-1.compute.internal.18e36ca023ef1ab9  default   4911 0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-18.us-west-1.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-140-18.us-west-1.compute.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-140-18.us-west-1.compute.internal,},FirstTimestamp:2026-10-30 21:41:51 +0000 UTC,LastTimestamp:2026-10-30 21:41:51.826812054 +0000 UTC m=+0.518374783,Count:3,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-18.us-west-1.compute.internal,}"
Oct 30 21:41:52.593251 ip-10-0-140-18 kubenswrapper[2580]: E1030 21:41:52.593227    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-140-18.us-west-1.compute.internal\" not found"
Oct 30 21:41:52.693956 ip-10-0-140-18 kubenswrapper[2580]: E1030 21:41:52.693885    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-140-18.us-west-1.compute.internal\" not found"
Oct 30 21:41:52.794383 ip-10-0-140-18 kubenswrapper[2580]: E1030 21:41:52.794350    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-140-18.us-west-1.compute.internal\" not found"
Oct 30 21:41:52.894501 ip-10-0-140-18 kubenswrapper[2580]: E1030 21:41:52.894465    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-140-18.us-west-1.compute.internal\" not found"
Oct 30 21:41:52.899620 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:52.899602    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 21:41:52.900587 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:52.900570    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-18.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Oct 30 21:41:52.900587 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:52.900599    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-18.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 21:41:52.900711 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:52.900610    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-18.us-west-1.compute.internal" event="NodeHasSufficientPID"
Oct 30 21:41:52.912735 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:52.912709    2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-10-29 21:36:53 +0000 UTC" deadline="2028-06-01 03:08:28.052830171 +0000 UTC"
Oct 30 21:41:52.912795 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:52.912737    2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13901h26m35.140098009s"
Oct 30 21:41:52.995612 ip-10-0-140-18 kubenswrapper[2580]: E1030 21:41:52.995529    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-140-18.us-west-1.compute.internal\" not found"
Oct 30 21:41:53.094122 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:53.094091    2580 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 21:41:53.096205 ip-10-0-140-18 kubenswrapper[2580]: E1030 21:41:53.096189    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-140-18.us-west-1.compute.internal\" not found"
Oct 30 21:41:53.192006 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:53.191805    2580 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 21:41:53.196289 ip-10-0-140-18 kubenswrapper[2580]: E1030 21:41:53.196253    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-140-18.us-west-1.compute.internal\" not found"
Oct 30 21:41:53.296761 ip-10-0-140-18 kubenswrapper[2580]: E1030 21:41:53.296739    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-140-18.us-west-1.compute.internal\" not found"
Oct 30 21:41:53.306463 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:53.306426    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaae6ff4840f4ffd5842f280d0afa74f6.slice/crio-c829b82f78e3de9b1b8ccab3a669062ac59ed7b3e800a2ef83c3037dcc6b0b75 WatchSource:0}: Error finding container c829b82f78e3de9b1b8ccab3a669062ac59ed7b3e800a2ef83c3037dcc6b0b75: Status 404 returned error can't find the container with id c829b82f78e3de9b1b8ccab3a669062ac59ed7b3e800a2ef83c3037dcc6b0b75
Oct 30 21:41:53.306646 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:53.306633    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a45018bbca93ea1080adf030d8fbb7f.slice/crio-801bef426e6476fee21829f7a500535c139dc4114bb4ef9843db0c96de15297d WatchSource:0}: Error finding container 801bef426e6476fee21829f7a500535c139dc4114bb4ef9843db0c96de15297d: Status 404 returned error can't find the container with id 801bef426e6476fee21829f7a500535c139dc4114bb4ef9843db0c96de15297d
Oct 30 21:41:53.311933 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:53.311911    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 21:41:53.397528 ip-10-0-140-18 kubenswrapper[2580]: E1030 21:41:53.397489    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-140-18.us-west-1.compute.internal\" not found"
Oct 30 21:41:53.498075 ip-10-0-140-18 kubenswrapper[2580]: E1030 21:41:53.498048    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-140-18.us-west-1.compute.internal\" not found"
Oct 30 21:41:53.520996 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:53.520975    2580 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 21:41:53.556075 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:53.556020    2580 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-18.us-west-1.compute.internal"
Oct 30 21:41:55.193164 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.193135    2580 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 21:41:55.194072 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.194053    2580 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-18.us-west-1.compute.internal"
Oct 30 21:41:55.206241 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.206210    2580 apiserver.go:52] "Watching apiserver"
Oct 30 21:41:55.234419 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.234391    2580 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Oct 30 21:41:55.255630 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.255606    2580 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 21:41:55.307818 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.307791    2580 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Oct 30 21:41:55.308120 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.308100    2580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-dt7cq","kube-system/kube-apiserver-proxy-ip-10-0-140-18.us-west-1.compute.internal","openshift-multus/multus-additional-cni-plugins-kkr27","openshift-multus/multus-hs4mm","openshift-network-diagnostics/network-check-target-9st74","kube-system/konnectivity-agent-krcxf","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8lm5l","openshift-dns/node-resolver-46nkk","openshift-cluster-node-tuning-operator/tuned-92gv2","openshift-image-registry/node-ca-jwvtb","openshift-multus/network-metrics-daemon-8jvqh","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-18.us-west-1.compute.internal","openshift-network-operator/iptables-alerter-bgfld","openshift-ovn-kubernetes/ovnkube-node-w6d9j"]
Oct 30 21:41:55.311888 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.311873    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8lm5l"
Oct 30 21:41:55.313907 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.313878    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8jvqh"
Oct 30 21:41:55.314028 ip-10-0-140-18 kubenswrapper[2580]: E1030 21:41:55.313973    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8jvqh" podUID="ccc5af13-906a-469d-9ca5-09366d1a7e93"
Oct 30 21:41:55.315887 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.315875    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-kkr27"
Oct 30 21:41:55.318023 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.318001    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-hs4mm"
Oct 30 21:41:55.320273 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.320255    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-w6d9j"
Oct 30 21:41:55.324025 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.324006    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-krcxf"
Oct 30 21:41:55.326366 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.326350    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-92gv2"
Oct 30 21:41:55.328518 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.328503    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-bgfld"
Oct 30 21:41:55.330652 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.330630    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-46nkk"
Oct 30 21:41:55.332838 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.332821    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-jwvtb"
Oct 30 21:41:55.335067 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.335053    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dt7cq"
Oct 30 21:41:55.335143 ip-10-0-140-18 kubenswrapper[2580]: E1030 21:41:55.335118    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dt7cq" podUID="0032cb99-3a37-40c6-98cb-2d40ae4a9319"
Oct 30 21:41:55.337288 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.337272    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9st74"
Oct 30 21:41:55.337342 ip-10-0-140-18 kubenswrapper[2580]: E1030 21:41:55.337326    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9st74" podUID="10d2e5ae-c1c4-400c-8c67-3aebb6caaa6a"
Oct 30 21:41:55.347123 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.347102    2580 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 21:41:55.349150 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.349132    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/641ac65e-00db-4646-b03c-9e3cb0c70feb-run-openvswitch\") pod \"ovnkube-node-w6d9j\" (UID: \"641ac65e-00db-4646-b03c-9e3cb0c70feb\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6d9j"
Oct 30 21:41:55.349242 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.349161    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c67fe06a-8f45-48d0-9451-b9157f57369e-sys\") pod \"tuned-92gv2\" (UID: \"c67fe06a-8f45-48d0-9451-b9157f57369e\") " pod="openshift-cluster-node-tuning-operator/tuned-92gv2"
Oct 30 21:41:55.349242 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.349184    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c67fe06a-8f45-48d0-9451-b9157f57369e-tmp\") pod \"tuned-92gv2\" (UID: \"c67fe06a-8f45-48d0-9451-b9157f57369e\") " pod="openshift-cluster-node-tuning-operator/tuned-92gv2"
Oct 30 21:41:55.349348 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.349265    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/0032cb99-3a37-40c6-98cb-2d40ae4a9319-kubelet-config\") pod \"global-pull-secret-syncer-dt7cq\" (UID: \"0032cb99-3a37-40c6-98cb-2d40ae4a9319\") " pod="kube-system/global-pull-secret-syncer-dt7cq"
Oct 30 21:41:55.349348 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.349292    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/641ac65e-00db-4646-b03c-9e3cb0c70feb-env-overrides\") pod \"ovnkube-node-w6d9j\" (UID: \"641ac65e-00db-4646-b03c-9e3cb0c70feb\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6d9j"
Oct 30 21:41:55.349348 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.349319    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/641ac65e-00db-4646-b03c-9e3cb0c70feb-host-kubelet\") pod \"ovnkube-node-w6d9j\" (UID: \"641ac65e-00db-4646-b03c-9e3cb0c70feb\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6d9j"
Oct 30 21:41:55.349348 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.349343    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2428ae10-7385-4056-aabe-bdf1b974a36b-host\") pod \"node-ca-jwvtb\" (UID: \"2428ae10-7385-4056-aabe-bdf1b974a36b\") " pod="openshift-image-registry/node-ca-jwvtb"
Oct 30 21:41:55.349485 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.349386    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/03a62eeb-c318-4ce3-b6e9-1afba98920e9-multus-cni-dir\") pod \"multus-hs4mm\" (UID: \"03a62eeb-c318-4ce3-b6e9-1afba98920e9\") " pod="openshift-multus/multus-hs4mm"
Oct 30 21:41:55.349485 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.349411    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/03a62eeb-c318-4ce3-b6e9-1afba98920e9-multus-socket-dir-parent\") pod \"multus-hs4mm\" (UID: \"03a62eeb-c318-4ce3-b6e9-1afba98920e9\") " pod="openshift-multus/multus-hs4mm"
Oct 30 21:41:55.349485 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.349429    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/03a62eeb-c318-4ce3-b6e9-1afba98920e9-multus-daemon-config\") pod \"multus-hs4mm\" (UID: \"03a62eeb-c318-4ce3-b6e9-1afba98920e9\") " pod="openshift-multus/multus-hs4mm"
Oct 30 21:41:55.349485 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.349455    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/03a62eeb-c318-4ce3-b6e9-1afba98920e9-etc-kubernetes\") pod \"multus-hs4mm\" (UID: \"03a62eeb-c318-4ce3-b6e9-1afba98920e9\") " pod="openshift-multus/multus-hs4mm"
Oct 30 21:41:55.349485 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.349484    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9bq8j\" (UniqueName: \"kubernetes.io/projected/03a62eeb-c318-4ce3-b6e9-1afba98920e9-kube-api-access-9bq8j\") pod \"multus-hs4mm\" (UID: \"03a62eeb-c318-4ce3-b6e9-1afba98920e9\") " pod="openshift-multus/multus-hs4mm"
Oct 30 21:41:55.349708 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.349500    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c67fe06a-8f45-48d0-9451-b9157f57369e-etc-sysctl-d\") pod \"tuned-92gv2\" (UID: \"c67fe06a-8f45-48d0-9451-b9157f57369e\") " pod="openshift-cluster-node-tuning-operator/tuned-92gv2"
Oct 30 21:41:55.349708 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.349514    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c67fe06a-8f45-48d0-9451-b9157f57369e-etc-tuned\") pod \"tuned-92gv2\" (UID: \"c67fe06a-8f45-48d0-9451-b9157f57369e\") " pod="openshift-cluster-node-tuning-operator/tuned-92gv2"
Oct 30 21:41:55.349708 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.349548    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-htnvw\" (UniqueName: \"kubernetes.io/projected/2634eb5b-4ce8-4742-819a-616dcd33da39-kube-api-access-htnvw\") pod \"node-resolver-46nkk\" (UID: \"2634eb5b-4ce8-4742-819a-616dcd33da39\") " pod="openshift-dns/node-resolver-46nkk"
Oct 30 21:41:55.349708 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.349576    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/641ac65e-00db-4646-b03c-9e3cb0c70feb-run-ovn\") pod \"ovnkube-node-w6d9j\" (UID: \"641ac65e-00db-4646-b03c-9e3cb0c70feb\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6d9j"
Oct 30 21:41:55.349708 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.349599    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2634eb5b-4ce8-4742-819a-616dcd33da39-tmp-dir\") pod \"node-resolver-46nkk\" (UID: \"2634eb5b-4ce8-4742-819a-616dcd33da39\") " pod="openshift-dns/node-resolver-46nkk"
Oct 30 21:41:55.349708 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.349625    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0743c782-f5ad-4d3a-86e7-fc66563902c7-tuning-conf-dir\") pod \"multus-additional-cni-plugins-kkr27\" (UID: \"0743c782-f5ad-4d3a-86e7-fc66563902c7\") " pod="openshift-multus/multus-additional-cni-plugins-kkr27"
Oct 30 21:41:55.349708 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.349651    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/0743c782-f5ad-4d3a-86e7-fc66563902c7-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-kkr27\" (UID: \"0743c782-f5ad-4d3a-86e7-fc66563902c7\") " pod="openshift-multus/multus-additional-cni-plugins-kkr27"
Oct 30 21:41:55.349708 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.349675    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ccc5af13-906a-469d-9ca5-09366d1a7e93-metrics-certs\") pod \"network-metrics-daemon-8jvqh\" (UID: \"ccc5af13-906a-469d-9ca5-09366d1a7e93\") " pod="openshift-multus/network-metrics-daemon-8jvqh"
Oct 30 21:41:55.349708 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.349705    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/641ac65e-00db-4646-b03c-9e3cb0c70feb-ovn-node-metrics-cert\") pod \"ovnkube-node-w6d9j\" (UID: \"641ac65e-00db-4646-b03c-9e3cb0c70feb\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6d9j"
Oct 30 21:41:55.350023 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.349728    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9hm6j\" (UniqueName: \"kubernetes.io/projected/641ac65e-00db-4646-b03c-9e3cb0c70feb-kube-api-access-9hm6j\") pod \"ovnkube-node-w6d9j\" (UID: \"641ac65e-00db-4646-b03c-9e3cb0c70feb\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6d9j"
Oct 30 21:41:55.350023 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.349790    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/03a62eeb-c318-4ce3-b6e9-1afba98920e9-system-cni-dir\") pod \"multus-hs4mm\" (UID: \"03a62eeb-c318-4ce3-b6e9-1afba98920e9\") " pod="openshift-multus/multus-hs4mm"
Oct 30 21:41:55.350023 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.349823    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/03a62eeb-c318-4ce3-b6e9-1afba98920e9-cni-binary-copy\") pod \"multus-hs4mm\" (UID: \"03a62eeb-c318-4ce3-b6e9-1afba98920e9\") " pod="openshift-multus/multus-hs4mm"
Oct 30 21:41:55.350023 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.349867    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/03a62eeb-c318-4ce3-b6e9-1afba98920e9-host-run-k8s-cni-cncf-io\") pod \"multus-hs4mm\" (UID: \"03a62eeb-c318-4ce3-b6e9-1afba98920e9\") " pod="openshift-multus/multus-hs4mm"
Oct 30 21:41:55.350023 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.349893    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/641ac65e-00db-4646-b03c-9e3cb0c70feb-run-systemd\") pod \"ovnkube-node-w6d9j\" (UID: \"641ac65e-00db-4646-b03c-9e3cb0c70feb\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6d9j"
Oct 30 21:41:55.350023 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.349982    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/5de0b6c7-0d1c-4732-abce-361a46df75c3-registration-dir\") pod \"aws-ebs-csi-driver-node-8lm5l\" (UID: \"5de0b6c7-0d1c-4732-abce-361a46df75c3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8lm5l"
Oct 30 21:41:55.350023 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.350012    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/0743c782-f5ad-4d3a-86e7-fc66563902c7-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-kkr27\" (UID: \"0743c782-f5ad-4d3a-86e7-fc66563902c7\") " pod="openshift-multus/multus-additional-cni-plugins-kkr27"
Oct 30 21:41:55.350266 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.350037    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/03a62eeb-c318-4ce3-b6e9-1afba98920e9-host-run-multus-certs\") pod \"multus-hs4mm\" (UID: \"03a62eeb-c318-4ce3-b6e9-1afba98920e9\") " pod="openshift-multus/multus-hs4mm"
Oct 30 21:41:55.350266 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.350065    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/2634eb5b-4ce8-4742-819a-616dcd33da39-hosts-file\") pod \"node-resolver-46nkk\" (UID: \"2634eb5b-4ce8-4742-819a-616dcd33da39\") " pod="openshift-dns/node-resolver-46nkk"
Oct 30 21:41:55.350266 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.350087    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/641ac65e-00db-4646-b03c-9e3cb0c70feb-host-cni-netd\") pod \"ovnkube-node-w6d9j\" (UID: \"641ac65e-00db-4646-b03c-9e3cb0c70feb\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6d9j"
Oct 30 21:41:55.350266 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.350146    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/06552eb8-7379-40df-b31c-a133e646afb9-konnectivity-ca\") pod \"konnectivity-agent-krcxf\" (UID: \"06552eb8-7379-40df-b31c-a133e646afb9\") " pod="kube-system/konnectivity-agent-krcxf"
Oct 30 21:41:55.350266 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.350178    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c67fe06a-8f45-48d0-9451-b9157f57369e-etc-modprobe-d\") pod \"tuned-92gv2\" (UID: \"c67fe06a-8f45-48d0-9451-b9157f57369e\") " pod="openshift-cluster-node-tuning-operator/tuned-92gv2"
Oct 30 21:41:55.350266 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.350201    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c67fe06a-8f45-48d0-9451-b9157f57369e-host\") pod \"tuned-92gv2\" (UID: \"c67fe06a-8f45-48d0-9451-b9157f57369e\") " pod="openshift-cluster-node-tuning-operator/tuned-92gv2"
Oct 30 21:41:55.350266 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.350217    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/641ac65e-00db-4646-b03c-9e3cb0c70feb-systemd-units\") pod \"ovnkube-node-w6d9j\" (UID: \"641ac65e-00db-4646-b03c-9e3cb0c70feb\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6d9j"
Oct 30 21:41:55.350266 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.350239    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/641ac65e-00db-4646-b03c-9e3cb0c70feb-log-socket\") pod \"ovnkube-node-w6d9j\" (UID: \"641ac65e-00db-4646-b03c-9e3cb0c70feb\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6d9j"
Oct 30 21:41:55.350266 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.350261    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/641ac65e-00db-4646-b03c-9e3cb0c70feb-ovnkube-script-lib\") pod \"ovnkube-node-w6d9j\" (UID: \"641ac65e-00db-4646-b03c-9e3cb0c70feb\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6d9j"
Oct 30 21:41:55.350639 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.350286    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5xlxd\" (UniqueName: \"kubernetes.io/projected/2428ae10-7385-4056-aabe-bdf1b974a36b-kube-api-access-5xlxd\") pod \"node-ca-jwvtb\" (UID: \"2428ae10-7385-4056-aabe-bdf1b974a36b\") " pod="openshift-image-registry/node-ca-jwvtb"
Oct 30 21:41:55.350639 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.350309    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/03a62eeb-c318-4ce3-b6e9-1afba98920e9-host-var-lib-cni-multus\") pod \"multus-hs4mm\" (UID: \"03a62eeb-c318-4ce3-b6e9-1afba98920e9\") " pod="openshift-multus/multus-hs4mm"
Oct 30 21:41:55.350639 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.350333    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/719f98aa-6247-4973-8d2c-a91aaf276984-host-slash\") pod \"iptables-alerter-bgfld\" (UID: \"719f98aa-6247-4973-8d2c-a91aaf276984\") " pod="openshift-network-operator/iptables-alerter-bgfld"
Oct 30 21:41:55.350639 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.350354    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pn9mk\" (UniqueName: \"kubernetes.io/projected/719f98aa-6247-4973-8d2c-a91aaf276984-kube-api-access-pn9mk\") pod \"iptables-alerter-bgfld\" (UID: \"719f98aa-6247-4973-8d2c-a91aaf276984\") " pod="openshift-network-operator/iptables-alerter-bgfld"
Oct 30 21:41:55.350639 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.350392    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/06552eb8-7379-40df-b31c-a133e646afb9-agent-certs\") pod \"konnectivity-agent-krcxf\" (UID: \"06552eb8-7379-40df-b31c-a133e646afb9\") " pod="kube-system/konnectivity-agent-krcxf"
Oct 30 21:41:55.350639 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.350435    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0743c782-f5ad-4d3a-86e7-fc66563902c7-cni-binary-copy\") pod \"multus-additional-cni-plugins-kkr27\" (UID: \"0743c782-f5ad-4d3a-86e7-fc66563902c7\") " pod="openshift-multus/multus-additional-cni-plugins-kkr27"
Oct 30 21:41:55.350639 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.350465    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c67fe06a-8f45-48d0-9451-b9157f57369e-lib-modules\") pod \"tuned-92gv2\" (UID: \"c67fe06a-8f45-48d0-9451-b9157f57369e\") " pod="openshift-cluster-node-tuning-operator/tuned-92gv2"
Oct 30 21:41:55.350639 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.350504    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/641ac65e-00db-4646-b03c-9e3cb0c70feb-host-slash\") pod \"ovnkube-node-w6d9j\" (UID: \"641ac65e-00db-4646-b03c-9e3cb0c70feb\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6d9j"
Oct 30 21:41:55.350639 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.350530    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/641ac65e-00db-4646-b03c-9e3cb0c70feb-host-run-ovn-kubernetes\") pod \"ovnkube-node-w6d9j\" (UID: \"641ac65e-00db-4646-b03c-9e3cb0c70feb\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6d9j"
Oct 30 21:41:55.350639 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.350557    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/5de0b6c7-0d1c-4732-abce-361a46df75c3-kubelet-dir\") pod \"aws-ebs-csi-driver-node-8lm5l\" (UID: \"5de0b6c7-0d1c-4732-abce-361a46df75c3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8lm5l"
Oct 30 21:41:55.350639 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.350578    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/03a62eeb-c318-4ce3-b6e9-1afba98920e9-host-var-lib-kubelet\") pod \"multus-hs4mm\" (UID: \"03a62eeb-c318-4ce3-b6e9-1afba98920e9\") " pod="openshift-multus/multus-hs4mm"
Oct 30 21:41:55.350639 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.350612    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-64s8x\" (UniqueName: \"kubernetes.io/projected/c67fe06a-8f45-48d0-9451-b9157f57369e-kube-api-access-64s8x\") pod \"tuned-92gv2\" (UID: \"c67fe06a-8f45-48d0-9451-b9157f57369e\") " pod="openshift-cluster-node-tuning-operator/tuned-92gv2"
Oct 30 21:41:55.351058 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.350647    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0743c782-f5ad-4d3a-86e7-fc66563902c7-system-cni-dir\") pod \"multus-additional-cni-plugins-kkr27\" (UID: \"0743c782-f5ad-4d3a-86e7-fc66563902c7\") " pod="openshift-multus/multus-additional-cni-plugins-kkr27"
Oct 30 21:41:55.351058 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.350664    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r9bdf\" (UniqueName: \"kubernetes.io/projected/0743c782-f5ad-4d3a-86e7-fc66563902c7-kube-api-access-r9bdf\") pod \"multus-additional-cni-plugins-kkr27\" (UID: \"0743c782-f5ad-4d3a-86e7-fc66563902c7\") " pod="openshift-multus/multus-additional-cni-plugins-kkr27"
Oct 30 21:41:55.351058 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.350679    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c67fe06a-8f45-48d0-9451-b9157f57369e-etc-systemd\") pod \"tuned-92gv2\" (UID: \"c67fe06a-8f45-48d0-9451-b9157f57369e\") " pod="openshift-cluster-node-tuning-operator/tuned-92gv2"
Oct 30 21:41:55.351058 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.350693    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/641ac65e-00db-4646-b03c-9e3cb0c70feb-host-run-netns\") pod \"ovnkube-node-w6d9j\" (UID: \"641ac65e-00db-4646-b03c-9e3cb0c70feb\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6d9j"
Oct 30 21:41:55.351058 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.350707    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/2428ae10-7385-4056-aabe-bdf1b974a36b-serviceca\") pod \"node-ca-jwvtb\" (UID: \"2428ae10-7385-4056-aabe-bdf1b974a36b\") " pod="openshift-image-registry/node-ca-jwvtb"
Oct 30 21:41:55.351058 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.350720    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c67fe06a-8f45-48d0-9451-b9157f57369e-run\") pod \"tuned-92gv2\" (UID: \"c67fe06a-8f45-48d0-9451-b9157f57369e\") " pod="openshift-cluster-node-tuning-operator/tuned-92gv2"
Oct 30 21:41:55.351058 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.350742    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c67fe06a-8f45-48d0-9451-b9157f57369e-var-lib-kubelet\") pod \"tuned-92gv2\" (UID: \"c67fe06a-8f45-48d0-9451-b9157f57369e\") " pod="openshift-cluster-node-tuning-operator/tuned-92gv2"
Oct 30 21:41:55.351058 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.350760    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/719f98aa-6247-4973-8d2c-a91aaf276984-iptables-alerter-script\") pod \"iptables-alerter-bgfld\" (UID: \"719f98aa-6247-4973-8d2c-a91aaf276984\") " pod="openshift-network-operator/iptables-alerter-bgfld"
Oct 30 21:41:55.351058 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.350774    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0032cb99-3a37-40c6-98cb-2d40ae4a9319-original-pull-secret\") pod \"global-pull-secret-syncer-dt7cq\" (UID: \"0032cb99-3a37-40c6-98cb-2d40ae4a9319\") " pod="kube-system/global-pull-secret-syncer-dt7cq"
Oct 30 21:41:55.351058 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.350822    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/5de0b6c7-0d1c-4732-abce-361a46df75c3-socket-dir\") pod \"aws-ebs-csi-driver-node-8lm5l\" (UID: \"5de0b6c7-0d1c-4732-abce-361a46df75c3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8lm5l"
Oct 30 21:41:55.351058 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.350851    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/5de0b6c7-0d1c-4732-abce-361a46df75c3-device-dir\") pod \"aws-ebs-csi-driver-node-8lm5l\" (UID: \"5de0b6c7-0d1c-4732-abce-361a46df75c3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8lm5l"
Oct 30 21:41:55.351058 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.350882    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/03a62eeb-c318-4ce3-b6e9-1afba98920e9-host-run-netns\") pod \"multus-hs4mm\" (UID: \"03a62eeb-c318-4ce3-b6e9-1afba98920e9\") " pod="openshift-multus/multus-hs4mm"
Oct 30 21:41:55.351058 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.350912    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/0032cb99-3a37-40c6-98cb-2d40ae4a9319-dbus\") pod \"global-pull-secret-syncer-dt7cq\" (UID: \"0032cb99-3a37-40c6-98cb-2d40ae4a9319\") " pod="kube-system/global-pull-secret-syncer-dt7cq"
Oct 30 21:41:55.351058 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.350956    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rfxjl\" (UniqueName: \"kubernetes.io/projected/5de0b6c7-0d1c-4732-abce-361a46df75c3-kube-api-access-rfxjl\") pod \"aws-ebs-csi-driver-node-8lm5l\" (UID: \"5de0b6c7-0d1c-4732-abce-361a46df75c3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8lm5l"
Oct 30 21:41:55.351058 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.350979    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c67fe06a-8f45-48d0-9451-b9157f57369e-etc-sysctl-conf\") pod \"tuned-92gv2\" (UID: \"c67fe06a-8f45-48d0-9451-b9157f57369e\") " pod="openshift-cluster-node-tuning-operator/tuned-92gv2"
Oct 30 21:41:55.351058 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.351001    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b8x4j\" (UniqueName: \"kubernetes.io/projected/ccc5af13-906a-469d-9ca5-09366d1a7e93-kube-api-access-b8x4j\") pod \"network-metrics-daemon-8jvqh\" (UID: \"ccc5af13-906a-469d-9ca5-09366d1a7e93\") " pod="openshift-multus/network-metrics-daemon-8jvqh"
Oct 30 21:41:55.351507 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.351022    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/641ac65e-00db-4646-b03c-9e3cb0c70feb-var-lib-openvswitch\") pod \"ovnkube-node-w6d9j\" (UID: \"641ac65e-00db-4646-b03c-9e3cb0c70feb\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6d9j"
Oct 30 21:41:55.351507 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.351038    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/641ac65e-00db-4646-b03c-9e3cb0c70feb-node-log\") pod \"ovnkube-node-w6d9j\" (UID: \"641ac65e-00db-4646-b03c-9e3cb0c70feb\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6d9j"
Oct 30 21:41:55.351507 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.351052    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/641ac65e-00db-4646-b03c-9e3cb0c70feb-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-w6d9j\" (UID: \"641ac65e-00db-4646-b03c-9e3cb0c70feb\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6d9j"
Oct 30 21:41:55.351507 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.351067    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0743c782-f5ad-4d3a-86e7-fc66563902c7-cnibin\") pod \"multus-additional-cni-plugins-kkr27\" (UID: \"0743c782-f5ad-4d3a-86e7-fc66563902c7\") " pod="openshift-multus/multus-additional-cni-plugins-kkr27"
Oct 30 21:41:55.351507 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.351085    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/5de0b6c7-0d1c-4732-abce-361a46df75c3-sys-fs\") pod \"aws-ebs-csi-driver-node-8lm5l\" (UID: \"5de0b6c7-0d1c-4732-abce-361a46df75c3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8lm5l"
Oct 30 21:41:55.351507 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.351109    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/03a62eeb-c318-4ce3-b6e9-1afba98920e9-hostroot\") pod \"multus-hs4mm\" (UID: \"03a62eeb-c318-4ce3-b6e9-1afba98920e9\") " pod="openshift-multus/multus-hs4mm"
Oct 30 21:41:55.351507 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.351133    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/03a62eeb-c318-4ce3-b6e9-1afba98920e9-multus-conf-dir\") pod \"multus-hs4mm\" (UID: \"03a62eeb-c318-4ce3-b6e9-1afba98920e9\") " pod="openshift-multus/multus-hs4mm"
Oct 30 21:41:55.351507 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.351154    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c67fe06a-8f45-48d0-9451-b9157f57369e-etc-kubernetes\") pod \"tuned-92gv2\" (UID: \"c67fe06a-8f45-48d0-9451-b9157f57369e\") " pod="openshift-cluster-node-tuning-operator/tuned-92gv2"
Oct 30 21:41:55.351507 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.351175    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/641ac65e-00db-4646-b03c-9e3cb0c70feb-host-cni-bin\") pod \"ovnkube-node-w6d9j\" (UID: \"641ac65e-00db-4646-b03c-9e3cb0c70feb\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6d9j"
Oct 30 21:41:55.351507 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.351197    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/641ac65e-00db-4646-b03c-9e3cb0c70feb-ovnkube-config\") pod \"ovnkube-node-w6d9j\" (UID: \"641ac65e-00db-4646-b03c-9e3cb0c70feb\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6d9j"
Oct 30 21:41:55.351507 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.351222    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/5de0b6c7-0d1c-4732-abce-361a46df75c3-etc-selinux\") pod \"aws-ebs-csi-driver-node-8lm5l\" (UID: \"5de0b6c7-0d1c-4732-abce-361a46df75c3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8lm5l"
Oct 30 21:41:55.351507 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.351244    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/03a62eeb-c318-4ce3-b6e9-1afba98920e9-cnibin\") pod \"multus-hs4mm\" (UID: \"03a62eeb-c318-4ce3-b6e9-1afba98920e9\") " pod="openshift-multus/multus-hs4mm"
Oct 30 21:41:55.351507 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.351280    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/03a62eeb-c318-4ce3-b6e9-1afba98920e9-host-var-lib-cni-bin\") pod \"multus-hs4mm\" (UID: \"03a62eeb-c318-4ce3-b6e9-1afba98920e9\") " pod="openshift-multus/multus-hs4mm"
Oct 30 21:41:55.351507 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.351303    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c67fe06a-8f45-48d0-9451-b9157f57369e-etc-sysconfig\") pod \"tuned-92gv2\" (UID: \"c67fe06a-8f45-48d0-9451-b9157f57369e\") " pod="openshift-cluster-node-tuning-operator/tuned-92gv2"
Oct 30 21:41:55.351507 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.351325    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/641ac65e-00db-4646-b03c-9e3cb0c70feb-etc-openvswitch\") pod \"ovnkube-node-w6d9j\" (UID: \"641ac65e-00db-4646-b03c-9e3cb0c70feb\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6d9j"
Oct 30 21:41:55.364110 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.364087    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Oct 30 21:41:55.364110 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.364108    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-fxktb\""
Oct 30 21:41:55.378860 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.378812    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-18.us-west-1.compute.internal" event={"ID":"aae6ff4840f4ffd5842f280d0afa74f6","Type":"ContainerStarted","Data":"c829b82f78e3de9b1b8ccab3a669062ac59ed7b3e800a2ef83c3037dcc6b0b75"}
Oct 30 21:41:55.380241 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.380221    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-18.us-west-1.compute.internal" event={"ID":"9a45018bbca93ea1080adf030d8fbb7f","Type":"ContainerStarted","Data":"801bef426e6476fee21829f7a500535c139dc4114bb4ef9843db0c96de15297d"}
Oct 30 21:41:55.406682 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.406580    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Oct 30 21:41:55.406682 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.406640    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Oct 30 21:41:55.406682 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.406646    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Oct 30 21:41:55.406997 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.406801    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Oct 30 21:41:55.406997 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.406862    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Oct 30 21:41:55.406997 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.406906    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Oct 30 21:41:55.407137 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.407084    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Oct 30 21:41:55.407183 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.407139    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Oct 30 21:41:55.407298 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.407280    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Oct 30 21:41:55.407344 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.407332    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Oct 30 21:41:55.407390 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.407334    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Oct 30 21:41:55.409227 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.409204    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Oct 30 21:41:55.409329 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.409253    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Oct 30 21:41:55.409388 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.409336    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Oct 30 21:41:55.409504 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.409488    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Oct 30 21:41:55.409504 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.409499    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Oct 30 21:41:55.409639 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.409513    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Oct 30 21:41:55.409639 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.409522    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Oct 30 21:41:55.409639 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.409522    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Oct 30 21:41:55.409639 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.409592    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Oct 30 21:41:55.409639 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.409633    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Oct 30 21:41:55.409869 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.409672    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Oct 30 21:41:55.409869 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.409689    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Oct 30 21:41:55.409869 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.409804    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Oct 30 21:41:55.426439 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.426413    2580 reflector.go:446] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Oct 30 21:41:55.426540 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.426458    2580 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-cfjf9"
Oct 30 21:41:55.434803 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.434781    2580 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Oct 30 21:41:55.452444 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.452417    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0743c782-f5ad-4d3a-86e7-fc66563902c7-system-cni-dir\") pod \"multus-additional-cni-plugins-kkr27\" (UID: \"0743c782-f5ad-4d3a-86e7-fc66563902c7\") " pod="openshift-multus/multus-additional-cni-plugins-kkr27"
Oct 30 21:41:55.452548 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.452447    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r9bdf\" (UniqueName: \"kubernetes.io/projected/0743c782-f5ad-4d3a-86e7-fc66563902c7-kube-api-access-r9bdf\") pod \"multus-additional-cni-plugins-kkr27\" (UID: \"0743c782-f5ad-4d3a-86e7-fc66563902c7\") " pod="openshift-multus/multus-additional-cni-plugins-kkr27"
Oct 30 21:41:55.452548 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.452467    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c67fe06a-8f45-48d0-9451-b9157f57369e-etc-systemd\") pod \"tuned-92gv2\" (UID: \"c67fe06a-8f45-48d0-9451-b9157f57369e\") " pod="openshift-cluster-node-tuning-operator/tuned-92gv2"
Oct 30 21:41:55.452548 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.452485    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/641ac65e-00db-4646-b03c-9e3cb0c70feb-host-run-netns\") pod \"ovnkube-node-w6d9j\" (UID: \"641ac65e-00db-4646-b03c-9e3cb0c70feb\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6d9j"
Oct 30 21:41:55.452548 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.452502    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/2428ae10-7385-4056-aabe-bdf1b974a36b-serviceca\") pod \"node-ca-jwvtb\" (UID: \"2428ae10-7385-4056-aabe-bdf1b974a36b\") " pod="openshift-image-registry/node-ca-jwvtb"
Oct 30 21:41:55.452752 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.452562    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/641ac65e-00db-4646-b03c-9e3cb0c70feb-host-run-netns\") pod \"ovnkube-node-w6d9j\" (UID: \"641ac65e-00db-4646-b03c-9e3cb0c70feb\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6d9j"
Oct 30 21:41:55.452752 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.452564    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c67fe06a-8f45-48d0-9451-b9157f57369e-etc-systemd\") pod \"tuned-92gv2\" (UID: \"c67fe06a-8f45-48d0-9451-b9157f57369e\") " pod="openshift-cluster-node-tuning-operator/tuned-92gv2"
Oct 30 21:41:55.452752 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.452568    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0743c782-f5ad-4d3a-86e7-fc66563902c7-system-cni-dir\") pod \"multus-additional-cni-plugins-kkr27\" (UID: \"0743c782-f5ad-4d3a-86e7-fc66563902c7\") " pod="openshift-multus/multus-additional-cni-plugins-kkr27"
Oct 30 21:41:55.452752 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.452594    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c67fe06a-8f45-48d0-9451-b9157f57369e-run\") pod \"tuned-92gv2\" (UID: \"c67fe06a-8f45-48d0-9451-b9157f57369e\") " pod="openshift-cluster-node-tuning-operator/tuned-92gv2"
Oct 30 21:41:55.452752 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.452629    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c67fe06a-8f45-48d0-9451-b9157f57369e-var-lib-kubelet\") pod \"tuned-92gv2\" (UID: \"c67fe06a-8f45-48d0-9451-b9157f57369e\") " pod="openshift-cluster-node-tuning-operator/tuned-92gv2"
Oct 30 21:41:55.452752 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.452660    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/719f98aa-6247-4973-8d2c-a91aaf276984-iptables-alerter-script\") pod \"iptables-alerter-bgfld\" (UID: \"719f98aa-6247-4973-8d2c-a91aaf276984\") " pod="openshift-network-operator/iptables-alerter-bgfld"
Oct 30 21:41:55.452752 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.452686    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0032cb99-3a37-40c6-98cb-2d40ae4a9319-original-pull-secret\") pod \"global-pull-secret-syncer-dt7cq\" (UID: \"0032cb99-3a37-40c6-98cb-2d40ae4a9319\") " pod="kube-system/global-pull-secret-syncer-dt7cq"
Oct 30 21:41:55.452752 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.452697    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c67fe06a-8f45-48d0-9451-b9157f57369e-var-lib-kubelet\") pod \"tuned-92gv2\" (UID: \"c67fe06a-8f45-48d0-9451-b9157f57369e\") " pod="openshift-cluster-node-tuning-operator/tuned-92gv2"
Oct 30 21:41:55.452752 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.452713    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/5de0b6c7-0d1c-4732-abce-361a46df75c3-socket-dir\") pod \"aws-ebs-csi-driver-node-8lm5l\" (UID: \"5de0b6c7-0d1c-4732-abce-361a46df75c3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8lm5l"
Oct 30 21:41:55.452752 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.452737    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/5de0b6c7-0d1c-4732-abce-361a46df75c3-device-dir\") pod \"aws-ebs-csi-driver-node-8lm5l\" (UID: \"5de0b6c7-0d1c-4732-abce-361a46df75c3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8lm5l"
Oct 30 21:41:55.452752 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.452743    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c67fe06a-8f45-48d0-9451-b9157f57369e-run\") pod \"tuned-92gv2\" (UID: \"c67fe06a-8f45-48d0-9451-b9157f57369e\") " pod="openshift-cluster-node-tuning-operator/tuned-92gv2"
Oct 30 21:41:55.453247 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.452760    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/03a62eeb-c318-4ce3-b6e9-1afba98920e9-host-run-netns\") pod \"multus-hs4mm\" (UID: \"03a62eeb-c318-4ce3-b6e9-1afba98920e9\") " pod="openshift-multus/multus-hs4mm"
Oct 30 21:41:55.453247 ip-10-0-140-18 kubenswrapper[2580]: E1030 21:41:55.452772    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 21:41:55.453247 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.452783    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/0032cb99-3a37-40c6-98cb-2d40ae4a9319-dbus\") pod \"global-pull-secret-syncer-dt7cq\" (UID: \"0032cb99-3a37-40c6-98cb-2d40ae4a9319\") " pod="kube-system/global-pull-secret-syncer-dt7cq"
Oct 30 21:41:55.453247 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.452809    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rfxjl\" (UniqueName: \"kubernetes.io/projected/5de0b6c7-0d1c-4732-abce-361a46df75c3-kube-api-access-rfxjl\") pod \"aws-ebs-csi-driver-node-8lm5l\" (UID: \"5de0b6c7-0d1c-4732-abce-361a46df75c3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8lm5l"
Oct 30 21:41:55.453247 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.452824    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/5de0b6c7-0d1c-4732-abce-361a46df75c3-device-dir\") pod \"aws-ebs-csi-driver-node-8lm5l\" (UID: \"5de0b6c7-0d1c-4732-abce-361a46df75c3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8lm5l"
Oct 30 21:41:55.453247 ip-10-0-140-18 kubenswrapper[2580]: E1030 21:41:55.452842    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0032cb99-3a37-40c6-98cb-2d40ae4a9319-original-pull-secret podName:0032cb99-3a37-40c6-98cb-2d40ae4a9319 nodeName:}" failed. No retries permitted until 2026-10-30 21:41:55.952819602 +0000 UTC m=+3.066761911 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0032cb99-3a37-40c6-98cb-2d40ae4a9319-original-pull-secret") pod "global-pull-secret-syncer-dt7cq" (UID: "0032cb99-3a37-40c6-98cb-2d40ae4a9319") : object "kube-system"/"original-pull-secret" not registered
Oct 30 21:41:55.453247 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.452860    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c67fe06a-8f45-48d0-9451-b9157f57369e-etc-sysctl-conf\") pod \"tuned-92gv2\" (UID: \"c67fe06a-8f45-48d0-9451-b9157f57369e\") " pod="openshift-cluster-node-tuning-operator/tuned-92gv2"
Oct 30 21:41:55.453247 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.452861    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/03a62eeb-c318-4ce3-b6e9-1afba98920e9-host-run-netns\") pod \"multus-hs4mm\" (UID: \"03a62eeb-c318-4ce3-b6e9-1afba98920e9\") " pod="openshift-multus/multus-hs4mm"
Oct 30 21:41:55.453247 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.452893    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/5de0b6c7-0d1c-4732-abce-361a46df75c3-socket-dir\") pod \"aws-ebs-csi-driver-node-8lm5l\" (UID: \"5de0b6c7-0d1c-4732-abce-361a46df75c3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8lm5l"
Oct 30 21:41:55.453247 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.452895    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b8x4j\" (UniqueName: \"kubernetes.io/projected/ccc5af13-906a-469d-9ca5-09366d1a7e93-kube-api-access-b8x4j\") pod \"network-metrics-daemon-8jvqh\" (UID: \"ccc5af13-906a-469d-9ca5-09366d1a7e93\") " pod="openshift-multus/network-metrics-daemon-8jvqh"
Oct 30 21:41:55.453247 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.452951    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/641ac65e-00db-4646-b03c-9e3cb0c70feb-var-lib-openvswitch\") pod \"ovnkube-node-w6d9j\" (UID: \"641ac65e-00db-4646-b03c-9e3cb0c70feb\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6d9j"
Oct 30 21:41:55.453247 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.452963    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/2428ae10-7385-4056-aabe-bdf1b974a36b-serviceca\") pod \"node-ca-jwvtb\" (UID: \"2428ae10-7385-4056-aabe-bdf1b974a36b\") " pod="openshift-image-registry/node-ca-jwvtb"
Oct 30 21:41:55.453247 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.452976    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/641ac65e-00db-4646-b03c-9e3cb0c70feb-node-log\") pod \"ovnkube-node-w6d9j\" (UID: \"641ac65e-00db-4646-b03c-9e3cb0c70feb\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6d9j"
Oct 30 21:41:55.453247 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.453007    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/641ac65e-00db-4646-b03c-9e3cb0c70feb-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-w6d9j\" (UID: \"641ac65e-00db-4646-b03c-9e3cb0c70feb\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6d9j"
Oct 30 21:41:55.453247 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.453025    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/0032cb99-3a37-40c6-98cb-2d40ae4a9319-dbus\") pod \"global-pull-secret-syncer-dt7cq\" (UID: \"0032cb99-3a37-40c6-98cb-2d40ae4a9319\") " pod="kube-system/global-pull-secret-syncer-dt7cq"
Oct 30 21:41:55.453247 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.453030    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c67fe06a-8f45-48d0-9451-b9157f57369e-etc-sysctl-conf\") pod \"tuned-92gv2\" (UID: \"c67fe06a-8f45-48d0-9451-b9157f57369e\") " pod="openshift-cluster-node-tuning-operator/tuned-92gv2"
Oct 30 21:41:55.453247 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.453033    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0743c782-f5ad-4d3a-86e7-fc66563902c7-cnibin\") pod \"multus-additional-cni-plugins-kkr27\" (UID: \"0743c782-f5ad-4d3a-86e7-fc66563902c7\") " pod="openshift-multus/multus-additional-cni-plugins-kkr27"
Oct 30 21:41:55.453865 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.453062    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/641ac65e-00db-4646-b03c-9e3cb0c70feb-node-log\") pod \"ovnkube-node-w6d9j\" (UID: \"641ac65e-00db-4646-b03c-9e3cb0c70feb\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6d9j"
Oct 30 21:41:55.453865 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.453103    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0743c782-f5ad-4d3a-86e7-fc66563902c7-cnibin\") pod \"multus-additional-cni-plugins-kkr27\" (UID: \"0743c782-f5ad-4d3a-86e7-fc66563902c7\") " pod="openshift-multus/multus-additional-cni-plugins-kkr27"
Oct 30 21:41:55.453865 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.453122    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/5de0b6c7-0d1c-4732-abce-361a46df75c3-sys-fs\") pod \"aws-ebs-csi-driver-node-8lm5l\" (UID: \"5de0b6c7-0d1c-4732-abce-361a46df75c3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8lm5l"
Oct 30 21:41:55.453865 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.453139    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/641ac65e-00db-4646-b03c-9e3cb0c70feb-var-lib-openvswitch\") pod \"ovnkube-node-w6d9j\" (UID: \"641ac65e-00db-4646-b03c-9e3cb0c70feb\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6d9j"
Oct 30 21:41:55.453865 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.453138    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/641ac65e-00db-4646-b03c-9e3cb0c70feb-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-w6d9j\" (UID: \"641ac65e-00db-4646-b03c-9e3cb0c70feb\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6d9j"
Oct 30 21:41:55.453865 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.453176    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/03a62eeb-c318-4ce3-b6e9-1afba98920e9-hostroot\") pod \"multus-hs4mm\" (UID: \"03a62eeb-c318-4ce3-b6e9-1afba98920e9\") " pod="openshift-multus/multus-hs4mm"
Oct 30 21:41:55.453865 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.453189    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/5de0b6c7-0d1c-4732-abce-361a46df75c3-sys-fs\") pod \"aws-ebs-csi-driver-node-8lm5l\" (UID: \"5de0b6c7-0d1c-4732-abce-361a46df75c3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8lm5l"
Oct 30 21:41:55.453865 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.453219    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/03a62eeb-c318-4ce3-b6e9-1afba98920e9-multus-conf-dir\") pod \"multus-hs4mm\" (UID: \"03a62eeb-c318-4ce3-b6e9-1afba98920e9\") " pod="openshift-multus/multus-hs4mm"
Oct 30 21:41:55.453865 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.453241    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/03a62eeb-c318-4ce3-b6e9-1afba98920e9-hostroot\") pod \"multus-hs4mm\" (UID: \"03a62eeb-c318-4ce3-b6e9-1afba98920e9\") " pod="openshift-multus/multus-hs4mm"
Oct 30 21:41:55.453865 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.453256    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/719f98aa-6247-4973-8d2c-a91aaf276984-iptables-alerter-script\") pod \"iptables-alerter-bgfld\" (UID: \"719f98aa-6247-4973-8d2c-a91aaf276984\") " pod="openshift-network-operator/iptables-alerter-bgfld"
Oct 30 21:41:55.453865 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.453289    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c67fe06a-8f45-48d0-9451-b9157f57369e-etc-kubernetes\") pod \"tuned-92gv2\" (UID: \"c67fe06a-8f45-48d0-9451-b9157f57369e\") " pod="openshift-cluster-node-tuning-operator/tuned-92gv2"
Oct 30 21:41:55.453865 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.453321    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c67fe06a-8f45-48d0-9451-b9157f57369e-etc-kubernetes\") pod \"tuned-92gv2\" (UID: \"c67fe06a-8f45-48d0-9451-b9157f57369e\") " pod="openshift-cluster-node-tuning-operator/tuned-92gv2"
Oct 30 21:41:55.453865 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.453344    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/641ac65e-00db-4646-b03c-9e3cb0c70feb-host-cni-bin\") pod \"ovnkube-node-w6d9j\" (UID: \"641ac65e-00db-4646-b03c-9e3cb0c70feb\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6d9j"
Oct 30 21:41:55.453865 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.453358    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/641ac65e-00db-4646-b03c-9e3cb0c70feb-ovnkube-config\") pod \"ovnkube-node-w6d9j\" (UID: \"641ac65e-00db-4646-b03c-9e3cb0c70feb\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6d9j"
Oct 30 21:41:55.453865 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.453399    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/5de0b6c7-0d1c-4732-abce-361a46df75c3-etc-selinux\") pod \"aws-ebs-csi-driver-node-8lm5l\" (UID: \"5de0b6c7-0d1c-4732-abce-361a46df75c3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8lm5l"
Oct 30 21:41:55.453865 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.453409    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/03a62eeb-c318-4ce3-b6e9-1afba98920e9-multus-conf-dir\") pod \"multus-hs4mm\" (UID: \"03a62eeb-c318-4ce3-b6e9-1afba98920e9\") " pod="openshift-multus/multus-hs4mm"
Oct 30 21:41:55.453865 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.453425    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/641ac65e-00db-4646-b03c-9e3cb0c70feb-host-cni-bin\") pod \"ovnkube-node-w6d9j\" (UID: \"641ac65e-00db-4646-b03c-9e3cb0c70feb\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6d9j"
Oct 30 21:41:55.453865 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.453428    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/03a62eeb-c318-4ce3-b6e9-1afba98920e9-cnibin\") pod \"multus-hs4mm\" (UID: \"03a62eeb-c318-4ce3-b6e9-1afba98920e9\") " pod="openshift-multus/multus-hs4mm"
Oct 30 21:41:55.454666 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.453502    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/5de0b6c7-0d1c-4732-abce-361a46df75c3-etc-selinux\") pod \"aws-ebs-csi-driver-node-8lm5l\" (UID: \"5de0b6c7-0d1c-4732-abce-361a46df75c3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8lm5l"
Oct 30 21:41:55.454666 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.453520    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/03a62eeb-c318-4ce3-b6e9-1afba98920e9-host-var-lib-cni-bin\") pod \"multus-hs4mm\" (UID: \"03a62eeb-c318-4ce3-b6e9-1afba98920e9\") " pod="openshift-multus/multus-hs4mm"
Oct 30 21:41:55.454666 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.453541    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/03a62eeb-c318-4ce3-b6e9-1afba98920e9-cnibin\") pod \"multus-hs4mm\" (UID: \"03a62eeb-c318-4ce3-b6e9-1afba98920e9\") " pod="openshift-multus/multus-hs4mm"
Oct 30 21:41:55.454666 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.453577    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c67fe06a-8f45-48d0-9451-b9157f57369e-etc-sysconfig\") pod \"tuned-92gv2\" (UID: \"c67fe06a-8f45-48d0-9451-b9157f57369e\") " pod="openshift-cluster-node-tuning-operator/tuned-92gv2"
Oct 30 21:41:55.454666 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.453578    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/03a62eeb-c318-4ce3-b6e9-1afba98920e9-host-var-lib-cni-bin\") pod \"multus-hs4mm\" (UID: \"03a62eeb-c318-4ce3-b6e9-1afba98920e9\") " pod="openshift-multus/multus-hs4mm"
Oct 30 21:41:55.454666 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.453603    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/641ac65e-00db-4646-b03c-9e3cb0c70feb-etc-openvswitch\") pod \"ovnkube-node-w6d9j\" (UID: \"641ac65e-00db-4646-b03c-9e3cb0c70feb\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6d9j"
Oct 30 21:41:55.454666 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.453620    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c67fe06a-8f45-48d0-9451-b9157f57369e-etc-sysconfig\") pod \"tuned-92gv2\" (UID: \"c67fe06a-8f45-48d0-9451-b9157f57369e\") " pod="openshift-cluster-node-tuning-operator/tuned-92gv2"
Oct 30 21:41:55.454666 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.453639    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/641ac65e-00db-4646-b03c-9e3cb0c70feb-run-openvswitch\") pod \"ovnkube-node-w6d9j\" (UID: \"641ac65e-00db-4646-b03c-9e3cb0c70feb\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6d9j"
Oct 30 21:41:55.454666 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.453643    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/641ac65e-00db-4646-b03c-9e3cb0c70feb-etc-openvswitch\") pod \"ovnkube-node-w6d9j\" (UID: \"641ac65e-00db-4646-b03c-9e3cb0c70feb\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6d9j"
Oct 30 21:41:55.454666 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.453668    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c67fe06a-8f45-48d0-9451-b9157f57369e-sys\") pod \"tuned-92gv2\" (UID: \"c67fe06a-8f45-48d0-9451-b9157f57369e\") " pod="openshift-cluster-node-tuning-operator/tuned-92gv2"
Oct 30 21:41:55.454666 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.453698    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/641ac65e-00db-4646-b03c-9e3cb0c70feb-run-openvswitch\") pod \"ovnkube-node-w6d9j\" (UID: \"641ac65e-00db-4646-b03c-9e3cb0c70feb\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6d9j"
Oct 30 21:41:55.454666 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.453737    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c67fe06a-8f45-48d0-9451-b9157f57369e-tmp\") pod \"tuned-92gv2\" (UID: \"c67fe06a-8f45-48d0-9451-b9157f57369e\") " pod="openshift-cluster-node-tuning-operator/tuned-92gv2"
Oct 30 21:41:55.454666 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.453772    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/641ac65e-00db-4646-b03c-9e3cb0c70feb-ovnkube-config\") pod \"ovnkube-node-w6d9j\" (UID: \"641ac65e-00db-4646-b03c-9e3cb0c70feb\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6d9j"
Oct 30 21:41:55.454666 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.453773    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c67fe06a-8f45-48d0-9451-b9157f57369e-sys\") pod \"tuned-92gv2\" (UID: \"c67fe06a-8f45-48d0-9451-b9157f57369e\") " pod="openshift-cluster-node-tuning-operator/tuned-92gv2"
Oct 30 21:41:55.454666 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.453793    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/0032cb99-3a37-40c6-98cb-2d40ae4a9319-kubelet-config\") pod \"global-pull-secret-syncer-dt7cq\" (UID: \"0032cb99-3a37-40c6-98cb-2d40ae4a9319\") " pod="kube-system/global-pull-secret-syncer-dt7cq"
Oct 30 21:41:55.454666 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.453810    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/641ac65e-00db-4646-b03c-9e3cb0c70feb-env-overrides\") pod \"ovnkube-node-w6d9j\" (UID: \"641ac65e-00db-4646-b03c-9e3cb0c70feb\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6d9j"
Oct 30 21:41:55.454666 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.453852    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/641ac65e-00db-4646-b03c-9e3cb0c70feb-host-kubelet\") pod \"ovnkube-node-w6d9j\" (UID: \"641ac65e-00db-4646-b03c-9e3cb0c70feb\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6d9j"
Oct 30 21:41:55.454666 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.453871    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/0032cb99-3a37-40c6-98cb-2d40ae4a9319-kubelet-config\") pod \"global-pull-secret-syncer-dt7cq\" (UID: \"0032cb99-3a37-40c6-98cb-2d40ae4a9319\") " pod="kube-system/global-pull-secret-syncer-dt7cq"
Oct 30 21:41:55.455519 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.453877    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2428ae10-7385-4056-aabe-bdf1b974a36b-host\") pod \"node-ca-jwvtb\" (UID: \"2428ae10-7385-4056-aabe-bdf1b974a36b\") " pod="openshift-image-registry/node-ca-jwvtb"
Oct 30 21:41:55.455519 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.453935    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2428ae10-7385-4056-aabe-bdf1b974a36b-host\") pod \"node-ca-jwvtb\" (UID: \"2428ae10-7385-4056-aabe-bdf1b974a36b\") " pod="openshift-image-registry/node-ca-jwvtb"
Oct 30 21:41:55.455519 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.453941    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/641ac65e-00db-4646-b03c-9e3cb0c70feb-host-kubelet\") pod \"ovnkube-node-w6d9j\" (UID: \"641ac65e-00db-4646-b03c-9e3cb0c70feb\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6d9j"
Oct 30 21:41:55.455519 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.453967    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/03a62eeb-c318-4ce3-b6e9-1afba98920e9-multus-cni-dir\") pod \"multus-hs4mm\" (UID: \"03a62eeb-c318-4ce3-b6e9-1afba98920e9\") " pod="openshift-multus/multus-hs4mm"
Oct 30 21:41:55.455519 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.453996    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/03a62eeb-c318-4ce3-b6e9-1afba98920e9-multus-socket-dir-parent\") pod \"multus-hs4mm\" (UID: \"03a62eeb-c318-4ce3-b6e9-1afba98920e9\") " pod="openshift-multus/multus-hs4mm"
Oct 30 21:41:55.455519 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.454010    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/03a62eeb-c318-4ce3-b6e9-1afba98920e9-multus-daemon-config\") pod \"multus-hs4mm\" (UID: \"03a62eeb-c318-4ce3-b6e9-1afba98920e9\") " pod="openshift-multus/multus-hs4mm"
Oct 30 21:41:55.455519 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.454041    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/03a62eeb-c318-4ce3-b6e9-1afba98920e9-etc-kubernetes\") pod \"multus-hs4mm\" (UID: \"03a62eeb-c318-4ce3-b6e9-1afba98920e9\") " pod="openshift-multus/multus-hs4mm"
Oct 30 21:41:55.455519 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.454064    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/03a62eeb-c318-4ce3-b6e9-1afba98920e9-multus-socket-dir-parent\") pod \"multus-hs4mm\" (UID: \"03a62eeb-c318-4ce3-b6e9-1afba98920e9\") " pod="openshift-multus/multus-hs4mm"
Oct 30 21:41:55.455519 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.454064    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9bq8j\" (UniqueName: \"kubernetes.io/projected/03a62eeb-c318-4ce3-b6e9-1afba98920e9-kube-api-access-9bq8j\") pod \"multus-hs4mm\" (UID: \"03a62eeb-c318-4ce3-b6e9-1afba98920e9\") " pod="openshift-multus/multus-hs4mm"
Oct 30 21:41:55.455519 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.454098    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/03a62eeb-c318-4ce3-b6e9-1afba98920e9-multus-cni-dir\") pod \"multus-hs4mm\" (UID: \"03a62eeb-c318-4ce3-b6e9-1afba98920e9\") " pod="openshift-multus/multus-hs4mm"
Oct 30 21:41:55.455519 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.454108    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-js6zb\" (UniqueName: \"kubernetes.io/projected/10d2e5ae-c1c4-400c-8c67-3aebb6caaa6a-kube-api-access-js6zb\") pod \"network-check-target-9st74\" (UID: \"10d2e5ae-c1c4-400c-8c67-3aebb6caaa6a\") " pod="openshift-network-diagnostics/network-check-target-9st74"
Oct 30 21:41:55.455519 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.454119    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/03a62eeb-c318-4ce3-b6e9-1afba98920e9-etc-kubernetes\") pod \"multus-hs4mm\" (UID: \"03a62eeb-c318-4ce3-b6e9-1afba98920e9\") " pod="openshift-multus/multus-hs4mm"
Oct 30 21:41:55.455519 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.454137    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c67fe06a-8f45-48d0-9451-b9157f57369e-etc-sysctl-d\") pod \"tuned-92gv2\" (UID: \"c67fe06a-8f45-48d0-9451-b9157f57369e\") " pod="openshift-cluster-node-tuning-operator/tuned-92gv2"
Oct 30 21:41:55.455519 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.454154    2580 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Oct 30 21:41:55.455519 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.454167    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c67fe06a-8f45-48d0-9451-b9157f57369e-etc-tuned\") pod \"tuned-92gv2\" (UID: \"c67fe06a-8f45-48d0-9451-b9157f57369e\") " pod="openshift-cluster-node-tuning-operator/tuned-92gv2"
Oct 30 21:41:55.455519 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.454212    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-htnvw\" (UniqueName: \"kubernetes.io/projected/2634eb5b-4ce8-4742-819a-616dcd33da39-kube-api-access-htnvw\") pod \"node-resolver-46nkk\" (UID: \"2634eb5b-4ce8-4742-819a-616dcd33da39\") " pod="openshift-dns/node-resolver-46nkk"
Oct 30 21:41:55.455519 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.454241    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/641ac65e-00db-4646-b03c-9e3cb0c70feb-run-ovn\") pod \"ovnkube-node-w6d9j\" (UID: \"641ac65e-00db-4646-b03c-9e3cb0c70feb\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6d9j"
Oct 30 21:41:55.455519 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.454264    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2634eb5b-4ce8-4742-819a-616dcd33da39-tmp-dir\") pod \"node-resolver-46nkk\" (UID: \"2634eb5b-4ce8-4742-819a-616dcd33da39\") " pod="openshift-dns/node-resolver-46nkk"
Oct 30 21:41:55.456216 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.454288    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0743c782-f5ad-4d3a-86e7-fc66563902c7-tuning-conf-dir\") pod \"multus-additional-cni-plugins-kkr27\" (UID: \"0743c782-f5ad-4d3a-86e7-fc66563902c7\") " pod="openshift-multus/multus-additional-cni-plugins-kkr27"
Oct 30 21:41:55.456216 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.454306    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c67fe06a-8f45-48d0-9451-b9157f57369e-etc-sysctl-d\") pod \"tuned-92gv2\" (UID: \"c67fe06a-8f45-48d0-9451-b9157f57369e\") " pod="openshift-cluster-node-tuning-operator/tuned-92gv2"
Oct 30 21:41:55.456216 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.454311    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/641ac65e-00db-4646-b03c-9e3cb0c70feb-run-ovn\") pod \"ovnkube-node-w6d9j\" (UID: \"641ac65e-00db-4646-b03c-9e3cb0c70feb\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6d9j"
Oct 30 21:41:55.456216 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.454314    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/0743c782-f5ad-4d3a-86e7-fc66563902c7-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-kkr27\" (UID: \"0743c782-f5ad-4d3a-86e7-fc66563902c7\") " pod="openshift-multus/multus-additional-cni-plugins-kkr27"
Oct 30 21:41:55.456216 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.454340    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/641ac65e-00db-4646-b03c-9e3cb0c70feb-env-overrides\") pod \"ovnkube-node-w6d9j\" (UID: \"641ac65e-00db-4646-b03c-9e3cb0c70feb\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6d9j"
Oct 30 21:41:55.456216 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.454362    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ccc5af13-906a-469d-9ca5-09366d1a7e93-metrics-certs\") pod \"network-metrics-daemon-8jvqh\" (UID: \"ccc5af13-906a-469d-9ca5-09366d1a7e93\") " pod="openshift-multus/network-metrics-daemon-8jvqh"
Oct 30 21:41:55.456216 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.454388    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/641ac65e-00db-4646-b03c-9e3cb0c70feb-ovn-node-metrics-cert\") pod \"ovnkube-node-w6d9j\" (UID: \"641ac65e-00db-4646-b03c-9e3cb0c70feb\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6d9j"
Oct 30 21:41:55.456216 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.454410    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9hm6j\" (UniqueName: \"kubernetes.io/projected/641ac65e-00db-4646-b03c-9e3cb0c70feb-kube-api-access-9hm6j\") pod \"ovnkube-node-w6d9j\" (UID: \"641ac65e-00db-4646-b03c-9e3cb0c70feb\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6d9j"
Oct 30 21:41:55.456216 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.454432    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/03a62eeb-c318-4ce3-b6e9-1afba98920e9-system-cni-dir\") pod \"multus-hs4mm\" (UID: \"03a62eeb-c318-4ce3-b6e9-1afba98920e9\") " pod="openshift-multus/multus-hs4mm"
Oct 30 21:41:55.456216 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.454465    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/03a62eeb-c318-4ce3-b6e9-1afba98920e9-cni-binary-copy\") pod \"multus-hs4mm\" (UID: \"03a62eeb-c318-4ce3-b6e9-1afba98920e9\") " pod="openshift-multus/multus-hs4mm"
Oct 30 21:41:55.456216 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.454501    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/03a62eeb-c318-4ce3-b6e9-1afba98920e9-multus-daemon-config\") pod \"multus-hs4mm\" (UID: \"03a62eeb-c318-4ce3-b6e9-1afba98920e9\") " pod="openshift-multus/multus-hs4mm"
Oct 30 21:41:55.456216 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.454502    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/03a62eeb-c318-4ce3-b6e9-1afba98920e9-host-run-k8s-cni-cncf-io\") pod \"multus-hs4mm\" (UID: \"03a62eeb-c318-4ce3-b6e9-1afba98920e9\") " pod="openshift-multus/multus-hs4mm"
Oct 30 21:41:55.456216 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.454538    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/641ac65e-00db-4646-b03c-9e3cb0c70feb-run-systemd\") pod \"ovnkube-node-w6d9j\" (UID: \"641ac65e-00db-4646-b03c-9e3cb0c70feb\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6d9j"
Oct 30 21:41:55.456216 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.454565    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/5de0b6c7-0d1c-4732-abce-361a46df75c3-registration-dir\") pod \"aws-ebs-csi-driver-node-8lm5l\" (UID: \"5de0b6c7-0d1c-4732-abce-361a46df75c3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8lm5l"
Oct 30 21:41:55.456216 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.454591    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/0743c782-f5ad-4d3a-86e7-fc66563902c7-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-kkr27\" (UID: \"0743c782-f5ad-4d3a-86e7-fc66563902c7\") " pod="openshift-multus/multus-additional-cni-plugins-kkr27"
Oct 30 21:41:55.456216 ip-10-0-140-18 kubenswrapper[2580]: E1030 21:41:55.454607    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 21:41:55.456216 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.454642    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2634eb5b-4ce8-4742-819a-616dcd33da39-tmp-dir\") pod \"node-resolver-46nkk\" (UID: \"2634eb5b-4ce8-4742-819a-616dcd33da39\") " pod="openshift-dns/node-resolver-46nkk"
Oct 30 21:41:55.456712 ip-10-0-140-18 kubenswrapper[2580]: E1030 21:41:55.454660    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ccc5af13-906a-469d-9ca5-09366d1a7e93-metrics-certs podName:ccc5af13-906a-469d-9ca5-09366d1a7e93 nodeName:}" failed. No retries permitted until 2026-10-30 21:41:55.954645859 +0000 UTC m=+3.068588153 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ccc5af13-906a-469d-9ca5-09366d1a7e93-metrics-certs") pod "network-metrics-daemon-8jvqh" (UID: "ccc5af13-906a-469d-9ca5-09366d1a7e93") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 21:41:55.456712 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.454641    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0743c782-f5ad-4d3a-86e7-fc66563902c7-tuning-conf-dir\") pod \"multus-additional-cni-plugins-kkr27\" (UID: \"0743c782-f5ad-4d3a-86e7-fc66563902c7\") " pod="openshift-multus/multus-additional-cni-plugins-kkr27"
Oct 30 21:41:55.456712 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.454643    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/03a62eeb-c318-4ce3-b6e9-1afba98920e9-host-run-multus-certs\") pod \"multus-hs4mm\" (UID: \"03a62eeb-c318-4ce3-b6e9-1afba98920e9\") " pod="openshift-multus/multus-hs4mm"
Oct 30 21:41:55.456712 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.454683    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/03a62eeb-c318-4ce3-b6e9-1afba98920e9-host-run-multus-certs\") pod \"multus-hs4mm\" (UID: \"03a62eeb-c318-4ce3-b6e9-1afba98920e9\") " pod="openshift-multus/multus-hs4mm"
Oct 30 21:41:55.456712 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.454707    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/2634eb5b-4ce8-4742-819a-616dcd33da39-hosts-file\") pod \"node-resolver-46nkk\" (UID: \"2634eb5b-4ce8-4742-819a-616dcd33da39\") " pod="openshift-dns/node-resolver-46nkk"
Oct 30 21:41:55.456712 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.454710    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/03a62eeb-c318-4ce3-b6e9-1afba98920e9-system-cni-dir\") pod \"multus-hs4mm\" (UID: \"03a62eeb-c318-4ce3-b6e9-1afba98920e9\") " pod="openshift-multus/multus-hs4mm"
Oct 30 21:41:55.456712 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.454757    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/2634eb5b-4ce8-4742-819a-616dcd33da39-hosts-file\") pod \"node-resolver-46nkk\" (UID: \"2634eb5b-4ce8-4742-819a-616dcd33da39\") " pod="openshift-dns/node-resolver-46nkk"
Oct 30 21:41:55.456712 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.454781    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/5de0b6c7-0d1c-4732-abce-361a46df75c3-registration-dir\") pod \"aws-ebs-csi-driver-node-8lm5l\" (UID: \"5de0b6c7-0d1c-4732-abce-361a46df75c3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8lm5l"
Oct 30 21:41:55.456712 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.454539    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/03a62eeb-c318-4ce3-b6e9-1afba98920e9-host-run-k8s-cni-cncf-io\") pod \"multus-hs4mm\" (UID: \"03a62eeb-c318-4ce3-b6e9-1afba98920e9\") " pod="openshift-multus/multus-hs4mm"
Oct 30 21:41:55.456712 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.454798    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/641ac65e-00db-4646-b03c-9e3cb0c70feb-run-systemd\") pod \"ovnkube-node-w6d9j\" (UID: \"641ac65e-00db-4646-b03c-9e3cb0c70feb\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6d9j"
Oct 30 21:41:55.456712 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.454826    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/0743c782-f5ad-4d3a-86e7-fc66563902c7-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-kkr27\" (UID: \"0743c782-f5ad-4d3a-86e7-fc66563902c7\") " pod="openshift-multus/multus-additional-cni-plugins-kkr27"
Oct 30 21:41:55.456712 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.454829    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/641ac65e-00db-4646-b03c-9e3cb0c70feb-host-cni-netd\") pod \"ovnkube-node-w6d9j\" (UID: \"641ac65e-00db-4646-b03c-9e3cb0c70feb\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6d9j"
Oct 30 21:41:55.456712 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.454853    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/641ac65e-00db-4646-b03c-9e3cb0c70feb-host-cni-netd\") pod \"ovnkube-node-w6d9j\" (UID: \"641ac65e-00db-4646-b03c-9e3cb0c70feb\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6d9j"
Oct 30 21:41:55.456712 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.454875    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/06552eb8-7379-40df-b31c-a133e646afb9-konnectivity-ca\") pod \"konnectivity-agent-krcxf\" (UID: \"06552eb8-7379-40df-b31c-a133e646afb9\") " pod="kube-system/konnectivity-agent-krcxf"
Oct 30 21:41:55.456712 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.454898    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c67fe06a-8f45-48d0-9451-b9157f57369e-etc-modprobe-d\") pod \"tuned-92gv2\" (UID: \"c67fe06a-8f45-48d0-9451-b9157f57369e\") " pod="openshift-cluster-node-tuning-operator/tuned-92gv2"
Oct 30 21:41:55.456712 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.454946    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c67fe06a-8f45-48d0-9451-b9157f57369e-host\") pod \"tuned-92gv2\" (UID: \"c67fe06a-8f45-48d0-9451-b9157f57369e\") " pod="openshift-cluster-node-tuning-operator/tuned-92gv2"
Oct 30 21:41:55.456712 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.454972    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/641ac65e-00db-4646-b03c-9e3cb0c70feb-systemd-units\") pod \"ovnkube-node-w6d9j\" (UID: \"641ac65e-00db-4646-b03c-9e3cb0c70feb\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6d9j"
Oct 30 21:41:55.457195 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.454995    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/641ac65e-00db-4646-b03c-9e3cb0c70feb-log-socket\") pod \"ovnkube-node-w6d9j\" (UID: \"641ac65e-00db-4646-b03c-9e3cb0c70feb\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6d9j"
Oct 30 21:41:55.457195 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.455019    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/641ac65e-00db-4646-b03c-9e3cb0c70feb-ovnkube-script-lib\") pod \"ovnkube-node-w6d9j\" (UID: \"641ac65e-00db-4646-b03c-9e3cb0c70feb\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6d9j"
Oct 30 21:41:55.457195 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.455023    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/641ac65e-00db-4646-b03c-9e3cb0c70feb-systemd-units\") pod \"ovnkube-node-w6d9j\" (UID: \"641ac65e-00db-4646-b03c-9e3cb0c70feb\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6d9j"
Oct 30 21:41:55.457195 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.455042    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5xlxd\" (UniqueName: \"kubernetes.io/projected/2428ae10-7385-4056-aabe-bdf1b974a36b-kube-api-access-5xlxd\") pod \"node-ca-jwvtb\" (UID: \"2428ae10-7385-4056-aabe-bdf1b974a36b\") " pod="openshift-image-registry/node-ca-jwvtb"
Oct 30 21:41:55.457195 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.455065    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c67fe06a-8f45-48d0-9451-b9157f57369e-host\") pod \"tuned-92gv2\" (UID: \"c67fe06a-8f45-48d0-9451-b9157f57369e\") " pod="openshift-cluster-node-tuning-operator/tuned-92gv2"
Oct 30 21:41:55.457195 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.455071    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/03a62eeb-c318-4ce3-b6e9-1afba98920e9-host-var-lib-cni-multus\") pod \"multus-hs4mm\" (UID: \"03a62eeb-c318-4ce3-b6e9-1afba98920e9\") " pod="openshift-multus/multus-hs4mm"
Oct 30 21:41:55.457195 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.455095    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/719f98aa-6247-4973-8d2c-a91aaf276984-host-slash\") pod \"iptables-alerter-bgfld\" (UID: \"719f98aa-6247-4973-8d2c-a91aaf276984\") " pod="openshift-network-operator/iptables-alerter-bgfld"
Oct 30 21:41:55.457195 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.455097    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/641ac65e-00db-4646-b03c-9e3cb0c70feb-log-socket\") pod \"ovnkube-node-w6d9j\" (UID: \"641ac65e-00db-4646-b03c-9e3cb0c70feb\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6d9j"
Oct 30 21:41:55.457195 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.455153    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/03a62eeb-c318-4ce3-b6e9-1afba98920e9-host-var-lib-cni-multus\") pod \"multus-hs4mm\" (UID: \"03a62eeb-c318-4ce3-b6e9-1afba98920e9\") " pod="openshift-multus/multus-hs4mm"
Oct 30 21:41:55.457195 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.454997    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c67fe06a-8f45-48d0-9451-b9157f57369e-etc-modprobe-d\") pod \"tuned-92gv2\" (UID: \"c67fe06a-8f45-48d0-9451-b9157f57369e\") " pod="openshift-cluster-node-tuning-operator/tuned-92gv2"
Oct 30 21:41:55.457195 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.455166    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/03a62eeb-c318-4ce3-b6e9-1afba98920e9-cni-binary-copy\") pod \"multus-hs4mm\" (UID: \"03a62eeb-c318-4ce3-b6e9-1afba98920e9\") " pod="openshift-multus/multus-hs4mm"
Oct 30 21:41:55.457195 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.455205    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/719f98aa-6247-4973-8d2c-a91aaf276984-host-slash\") pod \"iptables-alerter-bgfld\" (UID: \"719f98aa-6247-4973-8d2c-a91aaf276984\") " pod="openshift-network-operator/iptables-alerter-bgfld"
Oct 30 21:41:55.457195 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.455270    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pn9mk\" (UniqueName: \"kubernetes.io/projected/719f98aa-6247-4973-8d2c-a91aaf276984-kube-api-access-pn9mk\") pod \"iptables-alerter-bgfld\" (UID: \"719f98aa-6247-4973-8d2c-a91aaf276984\") " pod="openshift-network-operator/iptables-alerter-bgfld"
Oct 30 21:41:55.457195 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.455297    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/06552eb8-7379-40df-b31c-a133e646afb9-agent-certs\") pod \"konnectivity-agent-krcxf\" (UID: \"06552eb8-7379-40df-b31c-a133e646afb9\") " pod="kube-system/konnectivity-agent-krcxf"
Oct 30 21:41:55.457195 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.455312    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/0743c782-f5ad-4d3a-86e7-fc66563902c7-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-kkr27\" (UID: \"0743c782-f5ad-4d3a-86e7-fc66563902c7\") " pod="openshift-multus/multus-additional-cni-plugins-kkr27"
Oct 30 21:41:55.457195 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.455324    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0743c782-f5ad-4d3a-86e7-fc66563902c7-cni-binary-copy\") pod \"multus-additional-cni-plugins-kkr27\" (UID: \"0743c782-f5ad-4d3a-86e7-fc66563902c7\") " pod="openshift-multus/multus-additional-cni-plugins-kkr27"
Oct 30 21:41:55.457195 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.455349    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c67fe06a-8f45-48d0-9451-b9157f57369e-lib-modules\") pod \"tuned-92gv2\" (UID: \"c67fe06a-8f45-48d0-9451-b9157f57369e\") " pod="openshift-cluster-node-tuning-operator/tuned-92gv2"
Oct 30 21:41:55.457642 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.455398    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/641ac65e-00db-4646-b03c-9e3cb0c70feb-host-slash\") pod \"ovnkube-node-w6d9j\" (UID: \"641ac65e-00db-4646-b03c-9e3cb0c70feb\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6d9j"
Oct 30 21:41:55.457642 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.455551    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c67fe06a-8f45-48d0-9451-b9157f57369e-lib-modules\") pod \"tuned-92gv2\" (UID: \"c67fe06a-8f45-48d0-9451-b9157f57369e\") " pod="openshift-cluster-node-tuning-operator/tuned-92gv2"
Oct 30 21:41:55.457642 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.455562    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/641ac65e-00db-4646-b03c-9e3cb0c70feb-host-run-ovn-kubernetes\") pod \"ovnkube-node-w6d9j\" (UID: \"641ac65e-00db-4646-b03c-9e3cb0c70feb\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6d9j"
Oct 30 21:41:55.457642 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.455584    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/641ac65e-00db-4646-b03c-9e3cb0c70feb-host-slash\") pod \"ovnkube-node-w6d9j\" (UID: \"641ac65e-00db-4646-b03c-9e3cb0c70feb\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6d9j"
Oct 30 21:41:55.457642 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.455585    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/5de0b6c7-0d1c-4732-abce-361a46df75c3-kubelet-dir\") pod \"aws-ebs-csi-driver-node-8lm5l\" (UID: \"5de0b6c7-0d1c-4732-abce-361a46df75c3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8lm5l"
Oct 30 21:41:55.457642 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.455610    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/03a62eeb-c318-4ce3-b6e9-1afba98920e9-host-var-lib-kubelet\") pod \"multus-hs4mm\" (UID: \"03a62eeb-c318-4ce3-b6e9-1afba98920e9\") " pod="openshift-multus/multus-hs4mm"
Oct 30 21:41:55.457642 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.455624    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/641ac65e-00db-4646-b03c-9e3cb0c70feb-host-run-ovn-kubernetes\") pod \"ovnkube-node-w6d9j\" (UID: \"641ac65e-00db-4646-b03c-9e3cb0c70feb\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6d9j"
Oct 30 21:41:55.457642 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.455632    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-64s8x\" (UniqueName: \"kubernetes.io/projected/c67fe06a-8f45-48d0-9451-b9157f57369e-kube-api-access-64s8x\") pod \"tuned-92gv2\" (UID: \"c67fe06a-8f45-48d0-9451-b9157f57369e\") " pod="openshift-cluster-node-tuning-operator/tuned-92gv2"
Oct 30 21:41:55.457642 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.455682    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/5de0b6c7-0d1c-4732-abce-361a46df75c3-kubelet-dir\") pod \"aws-ebs-csi-driver-node-8lm5l\" (UID: \"5de0b6c7-0d1c-4732-abce-361a46df75c3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8lm5l"
Oct 30 21:41:55.457642 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.455692    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/03a62eeb-c318-4ce3-b6e9-1afba98920e9-host-var-lib-kubelet\") pod \"multus-hs4mm\" (UID: \"03a62eeb-c318-4ce3-b6e9-1afba98920e9\") " pod="openshift-multus/multus-hs4mm"
Oct 30 21:41:55.457642 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.456012    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/06552eb8-7379-40df-b31c-a133e646afb9-konnectivity-ca\") pod \"konnectivity-agent-krcxf\" (UID: \"06552eb8-7379-40df-b31c-a133e646afb9\") " pod="kube-system/konnectivity-agent-krcxf"
Oct 30 21:41:55.457642 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.456210    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0743c782-f5ad-4d3a-86e7-fc66563902c7-cni-binary-copy\") pod \"multus-additional-cni-plugins-kkr27\" (UID: \"0743c782-f5ad-4d3a-86e7-fc66563902c7\") " pod="openshift-multus/multus-additional-cni-plugins-kkr27"
Oct 30 21:41:55.457642 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.456286    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/641ac65e-00db-4646-b03c-9e3cb0c70feb-ovnkube-script-lib\") pod \"ovnkube-node-w6d9j\" (UID: \"641ac65e-00db-4646-b03c-9e3cb0c70feb\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6d9j"
Oct 30 21:41:55.457642 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.457558    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c67fe06a-8f45-48d0-9451-b9157f57369e-tmp\") pod \"tuned-92gv2\" (UID: \"c67fe06a-8f45-48d0-9451-b9157f57369e\") " pod="openshift-cluster-node-tuning-operator/tuned-92gv2"
Oct 30 21:41:55.457642 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.457590    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c67fe06a-8f45-48d0-9451-b9157f57369e-etc-tuned\") pod \"tuned-92gv2\" (UID: \"c67fe06a-8f45-48d0-9451-b9157f57369e\") " pod="openshift-cluster-node-tuning-operator/tuned-92gv2"
Oct 30 21:41:55.465954 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.465938    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Oct 30 21:41:55.466365 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.466346    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-qtzdm\""
Oct 30 21:41:55.466437 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.466422    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-6vmmq\""
Oct 30 21:41:55.466487 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.466459    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-xw9dh\""
Oct 30 21:41:55.466527 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.466508    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-2pvwl\""
Oct 30 21:41:55.466570 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.466526    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-k4c9l\""
Oct 30 21:41:55.466620 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.466576    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-rfsrz\""
Oct 30 21:41:55.466620 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.466613    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-zpjmc\""
Oct 30 21:41:55.466691 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.466613    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-qdnct\""
Oct 30 21:41:55.466691 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.466635    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Oct 30 21:41:55.477098 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.477070    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/641ac65e-00db-4646-b03c-9e3cb0c70feb-ovn-node-metrics-cert\") pod \"ovnkube-node-w6d9j\" (UID: \"641ac65e-00db-4646-b03c-9e3cb0c70feb\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6d9j"
Oct 30 21:41:55.478311 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.478291    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/06552eb8-7379-40df-b31c-a133e646afb9-agent-certs\") pod \"konnectivity-agent-krcxf\" (UID: \"06552eb8-7379-40df-b31c-a133e646afb9\") " pod="kube-system/konnectivity-agent-krcxf"
Oct 30 21:41:55.512459 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.512423    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b8x4j\" (UniqueName: \"kubernetes.io/projected/ccc5af13-906a-469d-9ca5-09366d1a7e93-kube-api-access-b8x4j\") pod \"network-metrics-daemon-8jvqh\" (UID: \"ccc5af13-906a-469d-9ca5-09366d1a7e93\") " pod="openshift-multus/network-metrics-daemon-8jvqh"
Oct 30 21:41:55.512609 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.512486    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rfxjl\" (UniqueName: \"kubernetes.io/projected/5de0b6c7-0d1c-4732-abce-361a46df75c3-kube-api-access-rfxjl\") pod \"aws-ebs-csi-driver-node-8lm5l\" (UID: \"5de0b6c7-0d1c-4732-abce-361a46df75c3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8lm5l"
Oct 30 21:41:55.514102 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.514070    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-64s8x\" (UniqueName: \"kubernetes.io/projected/c67fe06a-8f45-48d0-9451-b9157f57369e-kube-api-access-64s8x\") pod \"tuned-92gv2\" (UID: \"c67fe06a-8f45-48d0-9451-b9157f57369e\") " pod="openshift-cluster-node-tuning-operator/tuned-92gv2"
Oct 30 21:41:55.514226 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.514179    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-htnvw\" (UniqueName: \"kubernetes.io/projected/2634eb5b-4ce8-4742-819a-616dcd33da39-kube-api-access-htnvw\") pod \"node-resolver-46nkk\" (UID: \"2634eb5b-4ce8-4742-819a-616dcd33da39\") " pod="openshift-dns/node-resolver-46nkk"
Oct 30 21:41:55.514662 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.514633    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9hm6j\" (UniqueName: \"kubernetes.io/projected/641ac65e-00db-4646-b03c-9e3cb0c70feb-kube-api-access-9hm6j\") pod \"ovnkube-node-w6d9j\" (UID: \"641ac65e-00db-4646-b03c-9e3cb0c70feb\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6d9j"
Oct 30 21:41:55.514662 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.514655    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9bq8j\" (UniqueName: \"kubernetes.io/projected/03a62eeb-c318-4ce3-b6e9-1afba98920e9-kube-api-access-9bq8j\") pod \"multus-hs4mm\" (UID: \"03a62eeb-c318-4ce3-b6e9-1afba98920e9\") " pod="openshift-multus/multus-hs4mm"
Oct 30 21:41:55.514802 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.514782    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pn9mk\" (UniqueName: \"kubernetes.io/projected/719f98aa-6247-4973-8d2c-a91aaf276984-kube-api-access-pn9mk\") pod \"iptables-alerter-bgfld\" (UID: \"719f98aa-6247-4973-8d2c-a91aaf276984\") " pod="openshift-network-operator/iptables-alerter-bgfld"
Oct 30 21:41:55.514995 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.514980    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5xlxd\" (UniqueName: \"kubernetes.io/projected/2428ae10-7385-4056-aabe-bdf1b974a36b-kube-api-access-5xlxd\") pod \"node-ca-jwvtb\" (UID: \"2428ae10-7385-4056-aabe-bdf1b974a36b\") " pod="openshift-image-registry/node-ca-jwvtb"
Oct 30 21:41:55.515276 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.515256    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r9bdf\" (UniqueName: \"kubernetes.io/projected/0743c782-f5ad-4d3a-86e7-fc66563902c7-kube-api-access-r9bdf\") pod \"multus-additional-cni-plugins-kkr27\" (UID: \"0743c782-f5ad-4d3a-86e7-fc66563902c7\") " pod="openshift-multus/multus-additional-cni-plugins-kkr27"
Oct 30 21:41:55.556756 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.556712    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-js6zb\" (UniqueName: \"kubernetes.io/projected/10d2e5ae-c1c4-400c-8c67-3aebb6caaa6a-kube-api-access-js6zb\") pod \"network-check-target-9st74\" (UID: \"10d2e5ae-c1c4-400c-8c67-3aebb6caaa6a\") " pod="openshift-network-diagnostics/network-check-target-9st74"
Oct 30 21:41:55.611299 ip-10-0-140-18 kubenswrapper[2580]: E1030 21:41:55.611263    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 21:41:55.611299 ip-10-0-140-18 kubenswrapper[2580]: E1030 21:41:55.611289    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 21:41:55.611299 ip-10-0-140-18 kubenswrapper[2580]: E1030 21:41:55.611299    2580 projected.go:196] Error preparing data for projected volume kube-api-access-js6zb for pod openshift-network-diagnostics/network-check-target-9st74: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 21:41:55.611516 ip-10-0-140-18 kubenswrapper[2580]: E1030 21:41:55.611352    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/10d2e5ae-c1c4-400c-8c67-3aebb6caaa6a-kube-api-access-js6zb podName:10d2e5ae-c1c4-400c-8c67-3aebb6caaa6a nodeName:}" failed. No retries permitted until 2026-10-30 21:41:56.111337 +0000 UTC m=+3.225279291 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-js6zb" (UniqueName: "kubernetes.io/projected/10d2e5ae-c1c4-400c-8c67-3aebb6caaa6a-kube-api-access-js6zb") pod "network-check-target-9st74" (UID: "10d2e5ae-c1c4-400c-8c67-3aebb6caaa6a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 21:41:55.627493 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.627458    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8lm5l"
Oct 30 21:41:55.637422 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:55.637396    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5de0b6c7_0d1c_4732_abce_361a46df75c3.slice/crio-47636ce1c9aefb50eea1cbdaf7ddbc964ffdd48cb1b0c4160ebe0fc9c9f929e9 WatchSource:0}: Error finding container 47636ce1c9aefb50eea1cbdaf7ddbc964ffdd48cb1b0c4160ebe0fc9c9f929e9: Status 404 returned error can't find the container with id 47636ce1c9aefb50eea1cbdaf7ddbc964ffdd48cb1b0c4160ebe0fc9c9f929e9
Oct 30 21:41:55.640683 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.640667    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-hs4mm"
Oct 30 21:41:55.647599 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.647580    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-kkr27"
Oct 30 21:41:55.648628 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:55.648580    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod03a62eeb_c318_4ce3_b6e9_1afba98920e9.slice/crio-27f3fa2df241ed15c0c679361f882d1b7a1c504e1f510e4d568952e3f43917d3 WatchSource:0}: Error finding container 27f3fa2df241ed15c0c679361f882d1b7a1c504e1f510e4d568952e3f43917d3: Status 404 returned error can't find the container with id 27f3fa2df241ed15c0c679361f882d1b7a1c504e1f510e4d568952e3f43917d3
Oct 30 21:41:55.650665 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.650649    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-krcxf"
Oct 30 21:41:55.653093 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.653071    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-92gv2"
Oct 30 21:41:55.657286 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:55.657231    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0743c782_f5ad_4d3a_86e7_fc66563902c7.slice/crio-d8fa13f6ee6c04e40723c9b7bd7388ae0cb58ff50531dfd0e89bf94ad5309b77 WatchSource:0}: Error finding container d8fa13f6ee6c04e40723c9b7bd7388ae0cb58ff50531dfd0e89bf94ad5309b77: Status 404 returned error can't find the container with id d8fa13f6ee6c04e40723c9b7bd7388ae0cb58ff50531dfd0e89bf94ad5309b77
Oct 30 21:41:55.661675 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:55.661643    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06552eb8_7379_40df_b31c_a133e646afb9.slice/crio-79db91fb4dc34e8d30ddec93a6117fc1d4443038b0f2d8fc5c72483c4e7c1e68 WatchSource:0}: Error finding container 79db91fb4dc34e8d30ddec93a6117fc1d4443038b0f2d8fc5c72483c4e7c1e68: Status 404 returned error can't find the container with id 79db91fb4dc34e8d30ddec93a6117fc1d4443038b0f2d8fc5c72483c4e7c1e68
Oct 30 21:41:55.662285 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:55.662263    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc67fe06a_8f45_48d0_9451_b9157f57369e.slice/crio-ccd76436b40c03c67b556cc81abed6ea285e3f2803b98548bbd738c7461f5ea6 WatchSource:0}: Error finding container ccd76436b40c03c67b556cc81abed6ea285e3f2803b98548bbd738c7461f5ea6: Status 404 returned error can't find the container with id ccd76436b40c03c67b556cc81abed6ea285e3f2803b98548bbd738c7461f5ea6
Oct 30 21:41:55.662524 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.662506    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-bgfld"
Oct 30 21:41:55.665992 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.665977    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-46nkk"
Oct 30 21:41:55.670653 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.670633    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-w6d9j"
Oct 30 21:41:55.671656 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.671635    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-jwvtb"
Oct 30 21:41:55.672011 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:55.671992    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod719f98aa_6247_4973_8d2c_a91aaf276984.slice/crio-861ed8ec8169a70ec732075a3a81d7cce667a723e813db6aa2087d0be3b3bcf5 WatchSource:0}: Error finding container 861ed8ec8169a70ec732075a3a81d7cce667a723e813db6aa2087d0be3b3bcf5: Status 404 returned error can't find the container with id 861ed8ec8169a70ec732075a3a81d7cce667a723e813db6aa2087d0be3b3bcf5
Oct 30 21:41:55.675529 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:55.675507    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2634eb5b_4ce8_4742_819a_616dcd33da39.slice/crio-7b5fc3aef820c3083e5409913ee1bc84b01ce70c5542392f5f23ae7c3d241c67 WatchSource:0}: Error finding container 7b5fc3aef820c3083e5409913ee1bc84b01ce70c5542392f5f23ae7c3d241c67: Status 404 returned error can't find the container with id 7b5fc3aef820c3083e5409913ee1bc84b01ce70c5542392f5f23ae7c3d241c67
Oct 30 21:41:55.682335 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:55.682314    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod641ac65e_00db_4646_b03c_9e3cb0c70feb.slice/crio-ff246766b2304f22246fdd270a6b38453bc6d295d4e10b3536995b636b482eae WatchSource:0}: Error finding container ff246766b2304f22246fdd270a6b38453bc6d295d4e10b3536995b636b482eae: Status 404 returned error can't find the container with id ff246766b2304f22246fdd270a6b38453bc6d295d4e10b3536995b636b482eae
Oct 30 21:41:55.683172 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:41:55.683150    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2428ae10_7385_4056_aabe_bdf1b974a36b.slice/crio-20fb5e527b7a589fc2087d3651238f91f4c7354431ea9ec95b08f8d6e78d96c6 WatchSource:0}: Error finding container 20fb5e527b7a589fc2087d3651238f91f4c7354431ea9ec95b08f8d6e78d96c6: Status 404 returned error can't find the container with id 20fb5e527b7a589fc2087d3651238f91f4c7354431ea9ec95b08f8d6e78d96c6
Oct 30 21:41:55.959777 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.959689    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0032cb99-3a37-40c6-98cb-2d40ae4a9319-original-pull-secret\") pod \"global-pull-secret-syncer-dt7cq\" (UID: \"0032cb99-3a37-40c6-98cb-2d40ae4a9319\") " pod="kube-system/global-pull-secret-syncer-dt7cq"
Oct 30 21:41:55.959777 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:55.959765    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ccc5af13-906a-469d-9ca5-09366d1a7e93-metrics-certs\") pod \"network-metrics-daemon-8jvqh\" (UID: \"ccc5af13-906a-469d-9ca5-09366d1a7e93\") " pod="openshift-multus/network-metrics-daemon-8jvqh"
Oct 30 21:41:55.960028 ip-10-0-140-18 kubenswrapper[2580]: E1030 21:41:55.959844    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 21:41:55.960028 ip-10-0-140-18 kubenswrapper[2580]: E1030 21:41:55.959862    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 21:41:55.960028 ip-10-0-140-18 kubenswrapper[2580]: E1030 21:41:55.959934    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0032cb99-3a37-40c6-98cb-2d40ae4a9319-original-pull-secret podName:0032cb99-3a37-40c6-98cb-2d40ae4a9319 nodeName:}" failed. No retries permitted until 2026-10-30 21:41:56.959899106 +0000 UTC m=+4.073841398 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0032cb99-3a37-40c6-98cb-2d40ae4a9319-original-pull-secret") pod "global-pull-secret-syncer-dt7cq" (UID: "0032cb99-3a37-40c6-98cb-2d40ae4a9319") : object "kube-system"/"original-pull-secret" not registered
Oct 30 21:41:55.960028 ip-10-0-140-18 kubenswrapper[2580]: E1030 21:41:55.959964    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ccc5af13-906a-469d-9ca5-09366d1a7e93-metrics-certs podName:ccc5af13-906a-469d-9ca5-09366d1a7e93 nodeName:}" failed. No retries permitted until 2026-10-30 21:41:56.959944285 +0000 UTC m=+4.073886576 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ccc5af13-906a-469d-9ca5-09366d1a7e93-metrics-certs") pod "network-metrics-daemon-8jvqh" (UID: "ccc5af13-906a-469d-9ca5-09366d1a7e93") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 21:41:56.161803 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:56.161761    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-js6zb\" (UniqueName: \"kubernetes.io/projected/10d2e5ae-c1c4-400c-8c67-3aebb6caaa6a-kube-api-access-js6zb\") pod \"network-check-target-9st74\" (UID: \"10d2e5ae-c1c4-400c-8c67-3aebb6caaa6a\") " pod="openshift-network-diagnostics/network-check-target-9st74"
Oct 30 21:41:56.162032 ip-10-0-140-18 kubenswrapper[2580]: E1030 21:41:56.161979    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 21:41:56.162032 ip-10-0-140-18 kubenswrapper[2580]: E1030 21:41:56.162000    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 21:41:56.162032 ip-10-0-140-18 kubenswrapper[2580]: E1030 21:41:56.162014    2580 projected.go:196] Error preparing data for projected volume kube-api-access-js6zb for pod openshift-network-diagnostics/network-check-target-9st74: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 21:41:56.162161 ip-10-0-140-18 kubenswrapper[2580]: E1030 21:41:56.162132    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/10d2e5ae-c1c4-400c-8c67-3aebb6caaa6a-kube-api-access-js6zb podName:10d2e5ae-c1c4-400c-8c67-3aebb6caaa6a nodeName:}" failed. No retries permitted until 2026-10-30 21:41:57.162060957 +0000 UTC m=+4.276003267 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-js6zb" (UniqueName: "kubernetes.io/projected/10d2e5ae-c1c4-400c-8c67-3aebb6caaa6a-kube-api-access-js6zb") pod "network-check-target-9st74" (UID: "10d2e5ae-c1c4-400c-8c67-3aebb6caaa6a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 21:41:56.384397 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:56.384266    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kkr27" event={"ID":"0743c782-f5ad-4d3a-86e7-fc66563902c7","Type":"ContainerStarted","Data":"d8fa13f6ee6c04e40723c9b7bd7388ae0cb58ff50531dfd0e89bf94ad5309b77"}
Oct 30 21:41:56.385650 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:56.385619    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-hs4mm" event={"ID":"03a62eeb-c318-4ce3-b6e9-1afba98920e9","Type":"ContainerStarted","Data":"27f3fa2df241ed15c0c679361f882d1b7a1c504e1f510e4d568952e3f43917d3"}
Oct 30 21:41:56.387069 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:56.387043    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8lm5l" event={"ID":"5de0b6c7-0d1c-4732-abce-361a46df75c3","Type":"ContainerStarted","Data":"47636ce1c9aefb50eea1cbdaf7ddbc964ffdd48cb1b0c4160ebe0fc9c9f929e9"}
Oct 30 21:41:56.388773 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:56.388742    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-jwvtb" event={"ID":"2428ae10-7385-4056-aabe-bdf1b974a36b","Type":"ContainerStarted","Data":"20fb5e527b7a589fc2087d3651238f91f4c7354431ea9ec95b08f8d6e78d96c6"}
Oct 30 21:41:56.390450 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:56.390422    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-46nkk" event={"ID":"2634eb5b-4ce8-4742-819a-616dcd33da39","Type":"ContainerStarted","Data":"7b5fc3aef820c3083e5409913ee1bc84b01ce70c5542392f5f23ae7c3d241c67"}
Oct 30 21:41:56.391810 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:56.391783    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-92gv2" event={"ID":"c67fe06a-8f45-48d0-9451-b9157f57369e","Type":"ContainerStarted","Data":"ccd76436b40c03c67b556cc81abed6ea285e3f2803b98548bbd738c7461f5ea6"}
Oct 30 21:41:56.393179 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:56.393158    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-krcxf" event={"ID":"06552eb8-7379-40df-b31c-a133e646afb9","Type":"ContainerStarted","Data":"79db91fb4dc34e8d30ddec93a6117fc1d4443038b0f2d8fc5c72483c4e7c1e68"}
Oct 30 21:41:56.394774 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:56.394747    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-w6d9j" event={"ID":"641ac65e-00db-4646-b03c-9e3cb0c70feb","Type":"ContainerStarted","Data":"ff246766b2304f22246fdd270a6b38453bc6d295d4e10b3536995b636b482eae"}
Oct 30 21:41:56.396105 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:56.396084    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-bgfld" event={"ID":"719f98aa-6247-4973-8d2c-a91aaf276984","Type":"ContainerStarted","Data":"861ed8ec8169a70ec732075a3a81d7cce667a723e813db6aa2087d0be3b3bcf5"}
Oct 30 21:41:56.429615 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:56.429539    2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-29 21:36:55 +0000 UTC" deadline="2028-06-13 08:24:48.455460075 +0000 UTC"
Oct 30 21:41:56.429615 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:56.429609    2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14194h42m52.025854165s"
Oct 30 21:41:56.971022 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:56.969585    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ccc5af13-906a-469d-9ca5-09366d1a7e93-metrics-certs\") pod \"network-metrics-daemon-8jvqh\" (UID: \"ccc5af13-906a-469d-9ca5-09366d1a7e93\") " pod="openshift-multus/network-metrics-daemon-8jvqh"
Oct 30 21:41:56.971022 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:56.969649    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0032cb99-3a37-40c6-98cb-2d40ae4a9319-original-pull-secret\") pod \"global-pull-secret-syncer-dt7cq\" (UID: \"0032cb99-3a37-40c6-98cb-2d40ae4a9319\") " pod="kube-system/global-pull-secret-syncer-dt7cq"
Oct 30 21:41:56.971022 ip-10-0-140-18 kubenswrapper[2580]: E1030 21:41:56.969773    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 21:41:56.971022 ip-10-0-140-18 kubenswrapper[2580]: E1030 21:41:56.969829    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0032cb99-3a37-40c6-98cb-2d40ae4a9319-original-pull-secret podName:0032cb99-3a37-40c6-98cb-2d40ae4a9319 nodeName:}" failed. No retries permitted until 2026-10-30 21:41:58.969813308 +0000 UTC m=+6.083755600 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0032cb99-3a37-40c6-98cb-2d40ae4a9319-original-pull-secret") pod "global-pull-secret-syncer-dt7cq" (UID: "0032cb99-3a37-40c6-98cb-2d40ae4a9319") : object "kube-system"/"original-pull-secret" not registered
Oct 30 21:41:56.971022 ip-10-0-140-18 kubenswrapper[2580]: E1030 21:41:56.969977    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 21:41:56.971022 ip-10-0-140-18 kubenswrapper[2580]: E1030 21:41:56.970023    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ccc5af13-906a-469d-9ca5-09366d1a7e93-metrics-certs podName:ccc5af13-906a-469d-9ca5-09366d1a7e93 nodeName:}" failed. No retries permitted until 2026-10-30 21:41:58.97000942 +0000 UTC m=+6.083951716 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ccc5af13-906a-469d-9ca5-09366d1a7e93-metrics-certs") pod "network-metrics-daemon-8jvqh" (UID: "ccc5af13-906a-469d-9ca5-09366d1a7e93") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 21:41:57.172252 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:57.171568    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-js6zb\" (UniqueName: \"kubernetes.io/projected/10d2e5ae-c1c4-400c-8c67-3aebb6caaa6a-kube-api-access-js6zb\") pod \"network-check-target-9st74\" (UID: \"10d2e5ae-c1c4-400c-8c67-3aebb6caaa6a\") " pod="openshift-network-diagnostics/network-check-target-9st74"
Oct 30 21:41:57.172252 ip-10-0-140-18 kubenswrapper[2580]: E1030 21:41:57.171894    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 21:41:57.172252 ip-10-0-140-18 kubenswrapper[2580]: E1030 21:41:57.171914    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 21:41:57.172252 ip-10-0-140-18 kubenswrapper[2580]: E1030 21:41:57.171945    2580 projected.go:196] Error preparing data for projected volume kube-api-access-js6zb for pod openshift-network-diagnostics/network-check-target-9st74: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 21:41:57.172252 ip-10-0-140-18 kubenswrapper[2580]: E1030 21:41:57.172008    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/10d2e5ae-c1c4-400c-8c67-3aebb6caaa6a-kube-api-access-js6zb podName:10d2e5ae-c1c4-400c-8c67-3aebb6caaa6a nodeName:}" failed. No retries permitted until 2026-10-30 21:41:59.171987817 +0000 UTC m=+6.285930114 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-js6zb" (UniqueName: "kubernetes.io/projected/10d2e5ae-c1c4-400c-8c67-3aebb6caaa6a-kube-api-access-js6zb") pod "network-check-target-9st74" (UID: "10d2e5ae-c1c4-400c-8c67-3aebb6caaa6a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 21:41:57.378184 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:57.378152    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8jvqh"
Oct 30 21:41:57.378352 ip-10-0-140-18 kubenswrapper[2580]: E1030 21:41:57.378300    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8jvqh" podUID="ccc5af13-906a-469d-9ca5-09366d1a7e93"
Oct 30 21:41:57.378411 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:57.378392    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9st74"
Oct 30 21:41:57.378546 ip-10-0-140-18 kubenswrapper[2580]: E1030 21:41:57.378468    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9st74" podUID="10d2e5ae-c1c4-400c-8c67-3aebb6caaa6a"
Oct 30 21:41:57.378546 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:57.378531    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dt7cq"
Oct 30 21:41:57.378650 ip-10-0-140-18 kubenswrapper[2580]: E1030 21:41:57.378605    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dt7cq" podUID="0032cb99-3a37-40c6-98cb-2d40ae4a9319"
Oct 30 21:41:57.430074 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:57.430030    2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-29 21:36:55 +0000 UTC" deadline="2028-05-15 17:06:56.855645954 +0000 UTC"
Oct 30 21:41:57.430074 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:57.430072    2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13507h24m59.425578281s"
Oct 30 21:41:58.413067 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:58.413026    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-18.us-west-1.compute.internal" event={"ID":"aae6ff4840f4ffd5842f280d0afa74f6","Type":"ContainerStarted","Data":"f94328bfc9f11e4b93eac895167a7f89cea18e245973618e123f60d14ca12d17"}
Oct 30 21:41:58.427269 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:58.426601    2580 generic.go:356] "Generic (PLEG): container finished" podID="9a45018bbca93ea1080adf030d8fbb7f" containerID="655da74309b94247c02b665d6dd5306bf1c511226bfa3b60103fc150690d1d80" exitCode=0
Oct 30 21:41:58.427269 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:58.426654    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-18.us-west-1.compute.internal" event={"ID":"9a45018bbca93ea1080adf030d8fbb7f","Type":"ContainerDied","Data":"655da74309b94247c02b665d6dd5306bf1c511226bfa3b60103fc150690d1d80"}
Oct 30 21:41:58.532483 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:58.529426    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-18.us-west-1.compute.internal" podStartSLOduration=3.529407681 podStartE2EDuration="3.529407681s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 21:41:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:41:58.52915061 +0000 UTC m=+5.643092923" watchObservedRunningTime="2026-10-30 21:41:58.529407681 +0000 UTC m=+5.643349995"
Oct 30 21:41:58.987856 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:58.987407    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ccc5af13-906a-469d-9ca5-09366d1a7e93-metrics-certs\") pod \"network-metrics-daemon-8jvqh\" (UID: \"ccc5af13-906a-469d-9ca5-09366d1a7e93\") " pod="openshift-multus/network-metrics-daemon-8jvqh"
Oct 30 21:41:58.987856 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:58.987472    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0032cb99-3a37-40c6-98cb-2d40ae4a9319-original-pull-secret\") pod \"global-pull-secret-syncer-dt7cq\" (UID: \"0032cb99-3a37-40c6-98cb-2d40ae4a9319\") " pod="kube-system/global-pull-secret-syncer-dt7cq"
Oct 30 21:41:58.987856 ip-10-0-140-18 kubenswrapper[2580]: E1030 21:41:58.987598    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 21:41:58.987856 ip-10-0-140-18 kubenswrapper[2580]: E1030 21:41:58.987657    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0032cb99-3a37-40c6-98cb-2d40ae4a9319-original-pull-secret podName:0032cb99-3a37-40c6-98cb-2d40ae4a9319 nodeName:}" failed. No retries permitted until 2026-10-30 21:42:02.987641482 +0000 UTC m=+10.101583792 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0032cb99-3a37-40c6-98cb-2d40ae4a9319-original-pull-secret") pod "global-pull-secret-syncer-dt7cq" (UID: "0032cb99-3a37-40c6-98cb-2d40ae4a9319") : object "kube-system"/"original-pull-secret" not registered
Oct 30 21:41:58.987856 ip-10-0-140-18 kubenswrapper[2580]: E1030 21:41:58.987733    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 21:41:58.987856 ip-10-0-140-18 kubenswrapper[2580]: E1030 21:41:58.987766    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ccc5af13-906a-469d-9ca5-09366d1a7e93-metrics-certs podName:ccc5af13-906a-469d-9ca5-09366d1a7e93 nodeName:}" failed. No retries permitted until 2026-10-30 21:42:02.987753879 +0000 UTC m=+10.101696189 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ccc5af13-906a-469d-9ca5-09366d1a7e93-metrics-certs") pod "network-metrics-daemon-8jvqh" (UID: "ccc5af13-906a-469d-9ca5-09366d1a7e93") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 21:41:59.188888 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:59.188830    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-js6zb\" (UniqueName: \"kubernetes.io/projected/10d2e5ae-c1c4-400c-8c67-3aebb6caaa6a-kube-api-access-js6zb\") pod \"network-check-target-9st74\" (UID: \"10d2e5ae-c1c4-400c-8c67-3aebb6caaa6a\") " pod="openshift-network-diagnostics/network-check-target-9st74"
Oct 30 21:41:59.189108 ip-10-0-140-18 kubenswrapper[2580]: E1030 21:41:59.189089    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 21:41:59.189179 ip-10-0-140-18 kubenswrapper[2580]: E1030 21:41:59.189115    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 21:41:59.189179 ip-10-0-140-18 kubenswrapper[2580]: E1030 21:41:59.189128    2580 projected.go:196] Error preparing data for projected volume kube-api-access-js6zb for pod openshift-network-diagnostics/network-check-target-9st74: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 21:41:59.189276 ip-10-0-140-18 kubenswrapper[2580]: E1030 21:41:59.189187    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/10d2e5ae-c1c4-400c-8c67-3aebb6caaa6a-kube-api-access-js6zb podName:10d2e5ae-c1c4-400c-8c67-3aebb6caaa6a nodeName:}" failed. No retries permitted until 2026-10-30 21:42:03.189168429 +0000 UTC m=+10.303110742 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-js6zb" (UniqueName: "kubernetes.io/projected/10d2e5ae-c1c4-400c-8c67-3aebb6caaa6a-kube-api-access-js6zb") pod "network-check-target-9st74" (UID: "10d2e5ae-c1c4-400c-8c67-3aebb6caaa6a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 21:41:59.377525 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:59.377447    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dt7cq"
Oct 30 21:41:59.377658 ip-10-0-140-18 kubenswrapper[2580]: E1030 21:41:59.377568    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dt7cq" podUID="0032cb99-3a37-40c6-98cb-2d40ae4a9319"
Oct 30 21:41:59.377658 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:59.377651    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8jvqh"
Oct 30 21:41:59.377759 ip-10-0-140-18 kubenswrapper[2580]: E1030 21:41:59.377735    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8jvqh" podUID="ccc5af13-906a-469d-9ca5-09366d1a7e93"
Oct 30 21:41:59.377812 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:59.377796    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9st74"
Oct 30 21:41:59.377900 ip-10-0-140-18 kubenswrapper[2580]: E1030 21:41:59.377878    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9st74" podUID="10d2e5ae-c1c4-400c-8c67-3aebb6caaa6a"
Oct 30 21:41:59.435773 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:59.435736    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-18.us-west-1.compute.internal" event={"ID":"9a45018bbca93ea1080adf030d8fbb7f","Type":"ContainerStarted","Data":"ea286be6562ac7caf78ebe558a341016bcd7592f2dc2b2262b6369d191be2da1"}
Oct 30 21:41:59.441053 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:59.440988    2580 generic.go:356] "Generic (PLEG): container finished" podID="0743c782-f5ad-4d3a-86e7-fc66563902c7" containerID="e1b3266e356dd8341142ab24bc0f3fc3760ac6bf9b7520077225109599c6a4e6" exitCode=0
Oct 30 21:41:59.441507 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:59.441483    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kkr27" event={"ID":"0743c782-f5ad-4d3a-86e7-fc66563902c7","Type":"ContainerDied","Data":"e1b3266e356dd8341142ab24bc0f3fc3760ac6bf9b7520077225109599c6a4e6"}
Oct 30 21:41:59.553708 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:41:59.553644    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-18.us-west-1.compute.internal" podStartSLOduration=4.553624341 podStartE2EDuration="4.553624341s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 21:41:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:41:59.552807146 +0000 UTC m=+6.666749460" watchObservedRunningTime="2026-10-30 21:41:59.553624341 +0000 UTC m=+6.667566654"
Oct 30 21:42:00.446951 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:00.446902    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-krcxf" event={"ID":"06552eb8-7379-40df-b31c-a133e646afb9","Type":"ContainerStarted","Data":"61644c25fa1c94f14698bb65997051cd0d316b51809356bf97e5aca28dac71dd"}
Oct 30 21:42:00.560494 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:00.559909    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/konnectivity-agent-krcxf" podStartSLOduration=3.3938074719999998 podStartE2EDuration="7.559891751s" totalImagesPullingTime="4.166084279s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:41:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:42:00.559731828 +0000 UTC m=+7.673674143" watchObservedRunningTime="2026-10-30 21:42:00.559891751 +0000 UTC m=+7.673834067"
Oct 30 21:42:01.386657 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:01.385870    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9st74"
Oct 30 21:42:01.386657 ip-10-0-140-18 kubenswrapper[2580]: E1030 21:42:01.386060    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9st74" podUID="10d2e5ae-c1c4-400c-8c67-3aebb6caaa6a"
Oct 30 21:42:01.386657 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:01.386174    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dt7cq"
Oct 30 21:42:01.386657 ip-10-0-140-18 kubenswrapper[2580]: E1030 21:42:01.386301    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dt7cq" podUID="0032cb99-3a37-40c6-98cb-2d40ae4a9319"
Oct 30 21:42:01.386657 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:01.386379    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8jvqh"
Oct 30 21:42:01.386657 ip-10-0-140-18 kubenswrapper[2580]: E1030 21:42:01.386505    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8jvqh" podUID="ccc5af13-906a-469d-9ca5-09366d1a7e93"
Oct 30 21:42:01.451847 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:01.451775    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-jwvtb" event={"ID":"2428ae10-7385-4056-aabe-bdf1b974a36b","Type":"ContainerStarted","Data":"7f643255652252598290db2394b998ee8c406b233364c1d62684da1ae5cb65a8"}
Oct 30 21:42:02.459060 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:02.458984    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8lm5l" event={"ID":"5de0b6c7-0d1c-4732-abce-361a46df75c3","Type":"ContainerStarted","Data":"7ab01d18ceab30cd115fe89efc6c98d074eed914b54e63d177fd76a3edef3b7f"}
Oct 30 21:42:03.022464 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:03.022418    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0032cb99-3a37-40c6-98cb-2d40ae4a9319-original-pull-secret\") pod \"global-pull-secret-syncer-dt7cq\" (UID: \"0032cb99-3a37-40c6-98cb-2d40ae4a9319\") " pod="kube-system/global-pull-secret-syncer-dt7cq"
Oct 30 21:42:03.022653 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:03.022505    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ccc5af13-906a-469d-9ca5-09366d1a7e93-metrics-certs\") pod \"network-metrics-daemon-8jvqh\" (UID: \"ccc5af13-906a-469d-9ca5-09366d1a7e93\") " pod="openshift-multus/network-metrics-daemon-8jvqh"
Oct 30 21:42:03.022653 ip-10-0-140-18 kubenswrapper[2580]: E1030 21:42:03.022645    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 21:42:03.022772 ip-10-0-140-18 kubenswrapper[2580]: E1030 21:42:03.022709    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ccc5af13-906a-469d-9ca5-09366d1a7e93-metrics-certs podName:ccc5af13-906a-469d-9ca5-09366d1a7e93 nodeName:}" failed. No retries permitted until 2026-10-30 21:42:11.02269221 +0000 UTC m=+18.136634500 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ccc5af13-906a-469d-9ca5-09366d1a7e93-metrics-certs") pod "network-metrics-daemon-8jvqh" (UID: "ccc5af13-906a-469d-9ca5-09366d1a7e93") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 21:42:03.022833 ip-10-0-140-18 kubenswrapper[2580]: E1030 21:42:03.022778    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 21:42:03.022833 ip-10-0-140-18 kubenswrapper[2580]: E1030 21:42:03.022813    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0032cb99-3a37-40c6-98cb-2d40ae4a9319-original-pull-secret podName:0032cb99-3a37-40c6-98cb-2d40ae4a9319 nodeName:}" failed. No retries permitted until 2026-10-30 21:42:11.022800734 +0000 UTC m=+18.136743025 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0032cb99-3a37-40c6-98cb-2d40ae4a9319-original-pull-secret") pod "global-pull-secret-syncer-dt7cq" (UID: "0032cb99-3a37-40c6-98cb-2d40ae4a9319") : object "kube-system"/"original-pull-secret" not registered
Oct 30 21:42:03.224899 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:03.224863    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-js6zb\" (UniqueName: \"kubernetes.io/projected/10d2e5ae-c1c4-400c-8c67-3aebb6caaa6a-kube-api-access-js6zb\") pod \"network-check-target-9st74\" (UID: \"10d2e5ae-c1c4-400c-8c67-3aebb6caaa6a\") " pod="openshift-network-diagnostics/network-check-target-9st74"
Oct 30 21:42:03.225119 ip-10-0-140-18 kubenswrapper[2580]: E1030 21:42:03.225075    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 21:42:03.225119 ip-10-0-140-18 kubenswrapper[2580]: E1030 21:42:03.225097    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 21:42:03.225247 ip-10-0-140-18 kubenswrapper[2580]: E1030 21:42:03.225134    2580 projected.go:196] Error preparing data for projected volume kube-api-access-js6zb for pod openshift-network-diagnostics/network-check-target-9st74: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 21:42:03.225247 ip-10-0-140-18 kubenswrapper[2580]: E1030 21:42:03.225193    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/10d2e5ae-c1c4-400c-8c67-3aebb6caaa6a-kube-api-access-js6zb podName:10d2e5ae-c1c4-400c-8c67-3aebb6caaa6a nodeName:}" failed. No retries permitted until 2026-10-30 21:42:11.225176168 +0000 UTC m=+18.339118477 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-js6zb" (UniqueName: "kubernetes.io/projected/10d2e5ae-c1c4-400c-8c67-3aebb6caaa6a-kube-api-access-js6zb") pod "network-check-target-9st74" (UID: "10d2e5ae-c1c4-400c-8c67-3aebb6caaa6a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 21:42:03.271002 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:03.270949    2580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-krcxf"
Oct 30 21:42:03.271945 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:03.271910    2580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-krcxf"
Oct 30 21:42:03.375697 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:03.375624    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9st74"
Oct 30 21:42:03.375858 ip-10-0-140-18 kubenswrapper[2580]: E1030 21:42:03.375784    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9st74" podUID="10d2e5ae-c1c4-400c-8c67-3aebb6caaa6a"
Oct 30 21:42:03.375858 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:03.375847    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dt7cq"
Oct 30 21:42:03.376000 ip-10-0-140-18 kubenswrapper[2580]: E1030 21:42:03.375946    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dt7cq" podUID="0032cb99-3a37-40c6-98cb-2d40ae4a9319"
Oct 30 21:42:03.376000 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:03.375992    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8jvqh"
Oct 30 21:42:03.376110 ip-10-0-140-18 kubenswrapper[2580]: E1030 21:42:03.376070    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8jvqh" podUID="ccc5af13-906a-469d-9ca5-09366d1a7e93"
Oct 30 21:42:03.389006 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:03.388949    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-image-registry/node-ca-jwvtb" podStartSLOduration=4.975074032 podStartE2EDuration="10.388908035s" totalImagesPullingTime="5.413834003s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 21:41:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:42:01.568347772 +0000 UTC m=+8.682290087" watchObservedRunningTime="2026-10-30 21:42:03.388908035 +0000 UTC m=+10.502850349"
Oct 30 21:42:04.061234 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:04.061165    2580 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Oct 30 21:42:04.309455 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:04.309360    2580 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-10-30T21:42:04.061187205Z","UUID":"d61b54cf-cf19-4b25-bf30-4b09abc37c10","Handler":null,"Name":"","Endpoint":""}
Oct 30 21:42:04.311003 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:04.310867    2580 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Oct 30 21:42:04.311003 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:04.310910    2580 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Oct 30 21:42:04.466776 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:04.466733    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kkr27" event={"ID":"0743c782-f5ad-4d3a-86e7-fc66563902c7","Type":"ContainerStarted","Data":"1b8b63f9328443c9ce89909435c5e73442ca77b64a8fd94d708b5b2c192353a5"}
Oct 30 21:42:04.469283 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:04.469038    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-hs4mm" event={"ID":"03a62eeb-c318-4ce3-b6e9-1afba98920e9","Type":"ContainerStarted","Data":"c4e78297a17b58ceba5d3e3b44776208636ab45071d529a199c359c854da3374"}
Oct 30 21:42:04.472086 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:04.472013    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8lm5l" event={"ID":"5de0b6c7-0d1c-4732-abce-361a46df75c3","Type":"ContainerStarted","Data":"33bfc9405302d3f6bf4ec4765a87ffa0ccdc4a9b2a21639cf695afe1014b59a0"}
Oct 30 21:42:04.473989 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:04.473538    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-46nkk" event={"ID":"2634eb5b-4ce8-4742-819a-616dcd33da39","Type":"ContainerStarted","Data":"b8b18d7b04623f89f21ee64a5bf39d0804dde004b73f1878fe224734dd43b5f5"}
Oct 30 21:42:04.701553 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:04.701459    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-dns/node-resolver-46nkk" podStartSLOduration=3.071815222 podStartE2EDuration="11.701444584s" totalImagesPullingTime="8.629629362s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 21:41:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:42:04.701140835 +0000 UTC m=+11.815083147" watchObservedRunningTime="2026-10-30 21:42:04.701444584 +0000 UTC m=+11.815386896"
Oct 30 21:42:04.822989 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:04.822907    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-hs4mm" podStartSLOduration=3.378254004 podStartE2EDuration="11.822854952s" totalImagesPullingTime="8.444600948s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 21:41:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:42:04.822588487 +0000 UTC m=+11.936530802" watchObservedRunningTime="2026-10-30 21:42:04.822854952 +0000 UTC m=+11.936797265"
Oct 30 21:42:05.032133 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:05.032061    2580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-krcxf"
Oct 30 21:42:05.032269 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:05.032192    2580 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 30 21:42:05.033176 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:05.033118    2580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-krcxf"
Oct 30 21:42:05.378728 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:05.378698    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9st74"
Oct 30 21:42:05.379130 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:05.378781    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8jvqh"
Oct 30 21:42:05.379130 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:05.378879    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dt7cq"
Oct 30 21:42:05.379130 ip-10-0-140-18 kubenswrapper[2580]: E1030 21:42:05.378896    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9st74" podUID="10d2e5ae-c1c4-400c-8c67-3aebb6caaa6a"
Oct 30 21:42:05.379130 ip-10-0-140-18 kubenswrapper[2580]: E1030 21:42:05.378983    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dt7cq" podUID="0032cb99-3a37-40c6-98cb-2d40ae4a9319"
Oct 30 21:42:05.379130 ip-10-0-140-18 kubenswrapper[2580]: E1030 21:42:05.379088    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8jvqh" podUID="ccc5af13-906a-469d-9ca5-09366d1a7e93"
Oct 30 21:42:05.477110 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:05.477080    2580 generic.go:356] "Generic (PLEG): container finished" podID="0743c782-f5ad-4d3a-86e7-fc66563902c7" containerID="1b8b63f9328443c9ce89909435c5e73442ca77b64a8fd94d708b5b2c192353a5" exitCode=0
Oct 30 21:42:05.477257 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:05.477196    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kkr27" event={"ID":"0743c782-f5ad-4d3a-86e7-fc66563902c7","Type":"ContainerDied","Data":"1b8b63f9328443c9ce89909435c5e73442ca77b64a8fd94d708b5b2c192353a5"}
Oct 30 21:42:06.482268 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:06.482011    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8lm5l" event={"ID":"5de0b6c7-0d1c-4732-abce-361a46df75c3","Type":"ContainerStarted","Data":"b280c627fc82e49a7ea66a8231ea0fc6ef62c6e57fa2208bb07a334f6103e522"}
Oct 30 21:42:06.483940 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:06.483891    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-92gv2" event={"ID":"c67fe06a-8f45-48d0-9451-b9157f57369e","Type":"ContainerStarted","Data":"e7596d36139f8f1d0dbed44e7be8f68101fe6dafd572ca91800f76e1e4990f3b"}
Oct 30 21:42:06.485375 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:06.485351    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-bgfld" event={"ID":"719f98aa-6247-4973-8d2c-a91aaf276984","Type":"ContainerStarted","Data":"2aca3d97054939ee0b6cef7e634dc0ab778a83d856da3aab3c7a7e561040a06f"}
Oct 30 21:42:06.599024 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:06.598963    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8lm5l" podStartSLOduration=4.169913595 podStartE2EDuration="13.598949631s" totalImagesPullingTime="9.429036036s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 21:41:53 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:42:06.598618297 +0000 UTC m=+13.712560634" watchObservedRunningTime="2026-10-30 21:42:06.598949631 +0000 UTC m=+13.712891944"
Oct 30 21:42:06.724891 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:06.724845    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-92gv2" podStartSLOduration=3.41264806 podStartE2EDuration="13.72483027s" totalImagesPullingTime="10.31218221s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 21:41:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:42:06.724588375 +0000 UTC m=+13.838530688" watchObservedRunningTime="2026-10-30 21:42:06.72483027 +0000 UTC m=+13.838772584"
Oct 30 21:42:07.376087 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:07.376052    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8jvqh"
Oct 30 21:42:07.376265 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:07.376058    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dt7cq"
Oct 30 21:42:07.376265 ip-10-0-140-18 kubenswrapper[2580]: E1030 21:42:07.376194    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8jvqh" podUID="ccc5af13-906a-469d-9ca5-09366d1a7e93"
Oct 30 21:42:07.376265 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:07.376215    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9st74"
Oct 30 21:42:07.376394 ip-10-0-140-18 kubenswrapper[2580]: E1030 21:42:07.376314    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dt7cq" podUID="0032cb99-3a37-40c6-98cb-2d40ae4a9319"
Oct 30 21:42:07.376431 ip-10-0-140-18 kubenswrapper[2580]: E1030 21:42:07.376394    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9st74" podUID="10d2e5ae-c1c4-400c-8c67-3aebb6caaa6a"
Oct 30 21:42:07.490089 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:07.490043    2580 generic.go:356] "Generic (PLEG): container finished" podID="0743c782-f5ad-4d3a-86e7-fc66563902c7" containerID="31add3ceb278330ecedcba57b30a664d06a40dc12c8d453941a66cf981d33c4e" exitCode=0
Oct 30 21:42:07.490630 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:07.490150    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kkr27" event={"ID":"0743c782-f5ad-4d3a-86e7-fc66563902c7","Type":"ContainerDied","Data":"31add3ceb278330ecedcba57b30a664d06a40dc12c8d453941a66cf981d33c4e"}
Oct 30 21:42:07.612061 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:07.612007    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-bgfld" podStartSLOduration=6.180285881 podStartE2EDuration="14.611993165s" totalImagesPullingTime="8.431707284s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 21:41:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:42:06.840502212 +0000 UTC m=+13.954444526" watchObservedRunningTime="2026-10-30 21:42:07.611993165 +0000 UTC m=+14.725935487"
Oct 30 21:42:09.375273 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:09.375235    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dt7cq"
Oct 30 21:42:09.375273 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:09.375260    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9st74"
Oct 30 21:42:09.376083 ip-10-0-140-18 kubenswrapper[2580]: E1030 21:42:09.375408    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9st74" podUID="10d2e5ae-c1c4-400c-8c67-3aebb6caaa6a"
Oct 30 21:42:09.376083 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:09.375794    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8jvqh"
Oct 30 21:42:09.376083 ip-10-0-140-18 kubenswrapper[2580]: E1030 21:42:09.375906    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8jvqh" podUID="ccc5af13-906a-469d-9ca5-09366d1a7e93"
Oct 30 21:42:09.376083 ip-10-0-140-18 kubenswrapper[2580]: E1030 21:42:09.375986    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dt7cq" podUID="0032cb99-3a37-40c6-98cb-2d40ae4a9319"
Oct 30 21:42:09.498313 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:09.498280    2580 generic.go:356] "Generic (PLEG): container finished" podID="0743c782-f5ad-4d3a-86e7-fc66563902c7" containerID="0471160a86a336a9b462e092eee66ccd343b1e11033691c6220597d8aa122a3c" exitCode=0
Oct 30 21:42:09.498469 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:09.498342    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kkr27" event={"ID":"0743c782-f5ad-4d3a-86e7-fc66563902c7","Type":"ContainerDied","Data":"0471160a86a336a9b462e092eee66ccd343b1e11033691c6220597d8aa122a3c"}
Oct 30 21:42:11.088681 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:11.088468    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ccc5af13-906a-469d-9ca5-09366d1a7e93-metrics-certs\") pod \"network-metrics-daemon-8jvqh\" (UID: \"ccc5af13-906a-469d-9ca5-09366d1a7e93\") " pod="openshift-multus/network-metrics-daemon-8jvqh"
Oct 30 21:42:11.089107 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:11.088722    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0032cb99-3a37-40c6-98cb-2d40ae4a9319-original-pull-secret\") pod \"global-pull-secret-syncer-dt7cq\" (UID: \"0032cb99-3a37-40c6-98cb-2d40ae4a9319\") " pod="kube-system/global-pull-secret-syncer-dt7cq"
Oct 30 21:42:11.089107 ip-10-0-140-18 kubenswrapper[2580]: E1030 21:42:11.088577    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 21:42:11.089107 ip-10-0-140-18 kubenswrapper[2580]: E1030 21:42:11.088802    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ccc5af13-906a-469d-9ca5-09366d1a7e93-metrics-certs podName:ccc5af13-906a-469d-9ca5-09366d1a7e93 nodeName:}" failed. No retries permitted until 2026-10-30 21:42:27.088783382 +0000 UTC m=+34.202725677 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ccc5af13-906a-469d-9ca5-09366d1a7e93-metrics-certs") pod "network-metrics-daemon-8jvqh" (UID: "ccc5af13-906a-469d-9ca5-09366d1a7e93") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 21:42:11.089107 ip-10-0-140-18 kubenswrapper[2580]: E1030 21:42:11.088830    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 21:42:11.089107 ip-10-0-140-18 kubenswrapper[2580]: E1030 21:42:11.088875    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0032cb99-3a37-40c6-98cb-2d40ae4a9319-original-pull-secret podName:0032cb99-3a37-40c6-98cb-2d40ae4a9319 nodeName:}" failed. No retries permitted until 2026-10-30 21:42:27.088864126 +0000 UTC m=+34.202806433 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0032cb99-3a37-40c6-98cb-2d40ae4a9319-original-pull-secret") pod "global-pull-secret-syncer-dt7cq" (UID: "0032cb99-3a37-40c6-98cb-2d40ae4a9319") : object "kube-system"/"original-pull-secret" not registered
Oct 30 21:42:11.290382 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:11.290350    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-js6zb\" (UniqueName: \"kubernetes.io/projected/10d2e5ae-c1c4-400c-8c67-3aebb6caaa6a-kube-api-access-js6zb\") pod \"network-check-target-9st74\" (UID: \"10d2e5ae-c1c4-400c-8c67-3aebb6caaa6a\") " pod="openshift-network-diagnostics/network-check-target-9st74"
Oct 30 21:42:11.290568 ip-10-0-140-18 kubenswrapper[2580]: E1030 21:42:11.290545    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 21:42:11.290641 ip-10-0-140-18 kubenswrapper[2580]: E1030 21:42:11.290576    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 21:42:11.290641 ip-10-0-140-18 kubenswrapper[2580]: E1030 21:42:11.290592    2580 projected.go:196] Error preparing data for projected volume kube-api-access-js6zb for pod openshift-network-diagnostics/network-check-target-9st74: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 21:42:11.290745 ip-10-0-140-18 kubenswrapper[2580]: E1030 21:42:11.290650    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/10d2e5ae-c1c4-400c-8c67-3aebb6caaa6a-kube-api-access-js6zb podName:10d2e5ae-c1c4-400c-8c67-3aebb6caaa6a nodeName:}" failed. No retries permitted until 2026-10-30 21:42:27.290632703 +0000 UTC m=+34.404575005 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-js6zb" (UniqueName: "kubernetes.io/projected/10d2e5ae-c1c4-400c-8c67-3aebb6caaa6a-kube-api-access-js6zb") pod "network-check-target-9st74" (UID: "10d2e5ae-c1c4-400c-8c67-3aebb6caaa6a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 21:42:11.375429 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:11.375397    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dt7cq"
Oct 30 21:42:11.375429 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:11.375415    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8jvqh"
Oct 30 21:42:11.375643 ip-10-0-140-18 kubenswrapper[2580]: E1030 21:42:11.375554    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dt7cq" podUID="0032cb99-3a37-40c6-98cb-2d40ae4a9319"
Oct 30 21:42:11.375643 ip-10-0-140-18 kubenswrapper[2580]: E1030 21:42:11.375577    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8jvqh" podUID="ccc5af13-906a-469d-9ca5-09366d1a7e93"
Oct 30 21:42:11.375643 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:11.375625    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9st74"
Oct 30 21:42:11.375791 ip-10-0-140-18 kubenswrapper[2580]: E1030 21:42:11.375720    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9st74" podUID="10d2e5ae-c1c4-400c-8c67-3aebb6caaa6a"
Oct 30 21:42:11.506021 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:11.505760    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-w6d9j" event={"ID":"641ac65e-00db-4646-b03c-9e3cb0c70feb","Type":"ContainerStarted","Data":"d162e1fb95c931ff3617de89f51135b8a78168e592ed72cb4649f339cfd63615"}
Oct 30 21:42:11.506178 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:11.506086    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-w6d9j" event={"ID":"641ac65e-00db-4646-b03c-9e3cb0c70feb","Type":"ContainerStarted","Data":"b66a95106ec15c72f5f30a7206ad7976dac46901dbf89ddc4e62fc0b763d13ca"}
Oct 30 21:42:11.506178 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:11.506140    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-w6d9j" event={"ID":"641ac65e-00db-4646-b03c-9e3cb0c70feb","Type":"ContainerStarted","Data":"f563e400d5b4b704f9a2b57505912b69b75693efaf1ef8838f655bc4a074b7e6"}
Oct 30 21:42:11.506280 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:11.506186    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-w6d9j" event={"ID":"641ac65e-00db-4646-b03c-9e3cb0c70feb","Type":"ContainerStarted","Data":"86c10a25e1a44c5cf4d448da1f60778d29b1ec353944ddf7005d72f184402575"}
Oct 30 21:42:11.506280 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:11.506233    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-w6d9j" event={"ID":"641ac65e-00db-4646-b03c-9e3cb0c70feb","Type":"ContainerStarted","Data":"36a02780dfb1a4c9f1ca02dca22d97cc42ad61f9e2d26408c50c42db98ffcb40"}
Oct 30 21:42:11.506380 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:11.506280    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-w6d9j" event={"ID":"641ac65e-00db-4646-b03c-9e3cb0c70feb","Type":"ContainerStarted","Data":"df54bf06ef74ecd65b043142e05e5293ea9de054df075446b26bb928dc35e8a2"}
Oct 30 21:42:11.508737 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:11.508710    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kkr27" event={"ID":"0743c782-f5ad-4d3a-86e7-fc66563902c7","Type":"ContainerStarted","Data":"17b069cd8fc7a446971e3d4c429d877a859dc5d6e87ca3ad7826fed74e833bc6"}
Oct 30 21:42:12.514290 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:12.514259    2580 generic.go:356] "Generic (PLEG): container finished" podID="0743c782-f5ad-4d3a-86e7-fc66563902c7" containerID="17b069cd8fc7a446971e3d4c429d877a859dc5d6e87ca3ad7826fed74e833bc6" exitCode=0
Oct 30 21:42:12.514674 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:12.514309    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kkr27" event={"ID":"0743c782-f5ad-4d3a-86e7-fc66563902c7","Type":"ContainerDied","Data":"17b069cd8fc7a446971e3d4c429d877a859dc5d6e87ca3ad7826fed74e833bc6"}
Oct 30 21:42:13.376677 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:13.376637    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8jvqh"
Oct 30 21:42:13.376816 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:13.376797    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9st74"
Oct 30 21:42:13.376870 ip-10-0-140-18 kubenswrapper[2580]: E1030 21:42:13.376824    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8jvqh" podUID="ccc5af13-906a-469d-9ca5-09366d1a7e93"
Oct 30 21:42:13.376870 ip-10-0-140-18 kubenswrapper[2580]: E1030 21:42:13.376859    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9st74" podUID="10d2e5ae-c1c4-400c-8c67-3aebb6caaa6a"
Oct 30 21:42:13.376966 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:13.376905    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dt7cq"
Oct 30 21:42:13.377003 ip-10-0-140-18 kubenswrapper[2580]: E1030 21:42:13.376986    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dt7cq" podUID="0032cb99-3a37-40c6-98cb-2d40ae4a9319"
Oct 30 21:42:13.520206 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:13.520171    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-w6d9j" event={"ID":"641ac65e-00db-4646-b03c-9e3cb0c70feb","Type":"ContainerStarted","Data":"d3c388d89ab2c2046ba5daece1093a93c6a588c42f29c32c34421eff1991e7cd"}
Oct 30 21:42:13.523191 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:13.523165    2580 generic.go:356] "Generic (PLEG): container finished" podID="0743c782-f5ad-4d3a-86e7-fc66563902c7" containerID="0a7714b7c94a15bd15123619d730d6d97adea79a55948f28db83c43b8b9ad43e" exitCode=0
Oct 30 21:42:13.523314 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:13.523209    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kkr27" event={"ID":"0743c782-f5ad-4d3a-86e7-fc66563902c7","Type":"ContainerDied","Data":"0a7714b7c94a15bd15123619d730d6d97adea79a55948f28db83c43b8b9ad43e"}
Oct 30 21:42:14.528798 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:14.528766    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kkr27" event={"ID":"0743c782-f5ad-4d3a-86e7-fc66563902c7","Type":"ContainerStarted","Data":"b798e7bbf31a9ef59c04d8da8864f958c1e5d044d74c66bd83a86ef3fdf04114"}
Oct 30 21:42:14.646995 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:14.646947    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-kkr27" podStartSLOduration=9.432473603 podStartE2EDuration="21.646913869s" totalImagesPullingTime="12.214440266s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 21:41:53 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:42:14.646731673 +0000 UTC m=+21.760673985" watchObservedRunningTime="2026-10-30 21:42:14.646913869 +0000 UTC m=+21.760856182"
Oct 30 21:42:15.376051 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:15.376024    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dt7cq"
Oct 30 21:42:15.376224 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:15.376057    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9st74"
Oct 30 21:42:15.376224 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:15.376028    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8jvqh"
Oct 30 21:42:15.376224 ip-10-0-140-18 kubenswrapper[2580]: E1030 21:42:15.376127    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dt7cq" podUID="0032cb99-3a37-40c6-98cb-2d40ae4a9319"
Oct 30 21:42:15.376336 ip-10-0-140-18 kubenswrapper[2580]: E1030 21:42:15.376234    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8jvqh" podUID="ccc5af13-906a-469d-9ca5-09366d1a7e93"
Oct 30 21:42:15.376376 ip-10-0-140-18 kubenswrapper[2580]: E1030 21:42:15.376331    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9st74" podUID="10d2e5ae-c1c4-400c-8c67-3aebb6caaa6a"
Oct 30 21:42:15.534688 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:15.534648    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-w6d9j" event={"ID":"641ac65e-00db-4646-b03c-9e3cb0c70feb","Type":"ContainerStarted","Data":"9a296ab046273bd4511d611d9c42bc17f958aed98321bcd9f9cd8c7f70ed16ae"}
Oct 30 21:42:15.805200 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:15.805099    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-w6d9j" podStartSLOduration=7.5846209380000005 podStartE2EDuration="22.805085473s" totalImagesPullingTime="15.220464535s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 21:41:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:42:15.755253493 +0000 UTC m=+22.869195805" watchObservedRunningTime="2026-10-30 21:42:15.805085473 +0000 UTC m=+22.919027785"
Oct 30 21:42:16.536801 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:16.536775    2580 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 30 21:42:16.537399 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:16.537262    2580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-w6d9j"
Oct 30 21:42:16.537399 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:16.537335    2580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-w6d9j"
Oct 30 21:42:16.553398 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:16.553370    2580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-w6d9j"
Oct 30 21:42:16.553562 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:16.553532    2580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-w6d9j"
Oct 30 21:42:17.381671 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:17.381644    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dt7cq"
Oct 30 21:42:17.381845 ip-10-0-140-18 kubenswrapper[2580]: E1030 21:42:17.381733    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dt7cq" podUID="0032cb99-3a37-40c6-98cb-2d40ae4a9319"
Oct 30 21:42:17.381845 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:17.381829    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9st74"
Oct 30 21:42:17.381952 ip-10-0-140-18 kubenswrapper[2580]: E1030 21:42:17.381933    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9st74" podUID="10d2e5ae-c1c4-400c-8c67-3aebb6caaa6a"
Oct 30 21:42:17.382004 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:17.381982    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8jvqh"
Oct 30 21:42:17.382070 ip-10-0-140-18 kubenswrapper[2580]: E1030 21:42:17.382057    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8jvqh" podUID="ccc5af13-906a-469d-9ca5-09366d1a7e93"
Oct 30 21:42:17.539377 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:17.539342    2580 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 30 21:42:18.072028 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:18.071735    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-dt7cq"]
Oct 30 21:42:18.072157 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:18.072045    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dt7cq"
Oct 30 21:42:18.072157 ip-10-0-140-18 kubenswrapper[2580]: E1030 21:42:18.072127    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dt7cq" podUID="0032cb99-3a37-40c6-98cb-2d40ae4a9319"
Oct 30 21:42:18.075580 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:18.075550    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-9st74"]
Oct 30 21:42:18.075744 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:18.075656    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9st74"
Oct 30 21:42:18.075802 ip-10-0-140-18 kubenswrapper[2580]: E1030 21:42:18.075757    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9st74" podUID="10d2e5ae-c1c4-400c-8c67-3aebb6caaa6a"
Oct 30 21:42:18.076221 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:18.076195    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-8jvqh"]
Oct 30 21:42:18.076343 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:18.076281    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8jvqh"
Oct 30 21:42:18.076405 ip-10-0-140-18 kubenswrapper[2580]: E1030 21:42:18.076360    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8jvqh" podUID="ccc5af13-906a-469d-9ca5-09366d1a7e93"
Oct 30 21:42:18.541407 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:18.541382    2580 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 30 21:42:19.375163 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:19.375130    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9st74"
Oct 30 21:42:19.375362 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:19.375228    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dt7cq"
Oct 30 21:42:19.375362 ip-10-0-140-18 kubenswrapper[2580]: E1030 21:42:19.375335    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dt7cq" podUID="0032cb99-3a37-40c6-98cb-2d40ae4a9319"
Oct 30 21:42:19.375362 ip-10-0-140-18 kubenswrapper[2580]: E1030 21:42:19.375228    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9st74" podUID="10d2e5ae-c1c4-400c-8c67-3aebb6caaa6a"
Oct 30 21:42:20.376062 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:20.376028    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8jvqh"
Oct 30 21:42:20.376526 ip-10-0-140-18 kubenswrapper[2580]: E1030 21:42:20.376170    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8jvqh" podUID="ccc5af13-906a-469d-9ca5-09366d1a7e93"
Oct 30 21:42:21.378692 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:21.378658    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9st74"
Oct 30 21:42:21.379119 ip-10-0-140-18 kubenswrapper[2580]: E1030 21:42:21.378778    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9st74" podUID="10d2e5ae-c1c4-400c-8c67-3aebb6caaa6a"
Oct 30 21:42:21.379119 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:21.378818    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dt7cq"
Oct 30 21:42:21.379119 ip-10-0-140-18 kubenswrapper[2580]: E1030 21:42:21.378907    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dt7cq" podUID="0032cb99-3a37-40c6-98cb-2d40ae4a9319"
Oct 30 21:42:22.375143 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:22.375113    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8jvqh"
Oct 30 21:42:22.375297 ip-10-0-140-18 kubenswrapper[2580]: E1030 21:42:22.375226    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8jvqh" podUID="ccc5af13-906a-469d-9ca5-09366d1a7e93"
Oct 30 21:42:22.784026 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:22.783935    2580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-w6d9j"
Oct 30 21:42:22.784412 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:22.784127    2580 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 30 21:42:22.796158 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:22.796129    2580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-w6d9j"
Oct 30 21:42:23.378221 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:23.378185    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dt7cq"
Oct 30 21:42:23.378221 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:23.378214    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9st74"
Oct 30 21:42:23.378458 ip-10-0-140-18 kubenswrapper[2580]: E1030 21:42:23.378297    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dt7cq" podUID="0032cb99-3a37-40c6-98cb-2d40ae4a9319"
Oct 30 21:42:23.378514 ip-10-0-140-18 kubenswrapper[2580]: E1030 21:42:23.378447    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9st74" podUID="10d2e5ae-c1c4-400c-8c67-3aebb6caaa6a"
Oct 30 21:42:24.254565 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:24.254540    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-18.us-west-1.compute.internal" event="NodeReady"
Oct 30 21:42:24.255071 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:24.254693    2580 kubelet_node_status.go:477] "Fast updating node status as it just became ready"
Oct 30 21:42:24.373584 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:24.372406    2580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-5m6cr"]
Oct 30 21:42:24.376481 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:24.376460    2580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-hbmxt"]
Oct 30 21:42:24.376575 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:24.376557    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-5m6cr"
Oct 30 21:42:24.376751 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:24.376728    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8jvqh"
Oct 30 21:42:24.379333 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:24.379315    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-hbmxt"
Oct 30 21:42:24.433601 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:24.433553    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Oct 30 21:42:24.433849 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:24.433830    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Oct 30 21:42:24.433849 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:24.433845    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Oct 30 21:42:24.433947 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:24.433901    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Oct 30 21:42:24.434222 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:24.434208    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Oct 30 21:42:24.434306 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:24.434234    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Oct 30 21:42:24.434357 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:24.434336    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-6j5v7\""
Oct 30 21:42:24.434357 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:24.434341    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-tpd8h\""
Oct 30 21:42:24.434447 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:24.434386    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-spllf\""
Oct 30 21:42:24.441632 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:24.441607    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-5m6cr"]
Oct 30 21:42:24.481728 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:24.481697    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-hbmxt"]
Oct 30 21:42:24.496411 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:24.496373    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4436ec50-e190-4c1c-8b6c-2b9b5d12cb9f-cert\") pod \"ingress-canary-5m6cr\" (UID: \"4436ec50-e190-4c1c-8b6c-2b9b5d12cb9f\") " pod="openshift-ingress-canary/ingress-canary-5m6cr"
Oct 30 21:42:24.496411 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:24.496415    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f152360c-cf62-4d10-b2e5-b85ac7dfb739-tmp-dir\") pod \"dns-default-hbmxt\" (UID: \"f152360c-cf62-4d10-b2e5-b85ac7dfb739\") " pod="openshift-dns/dns-default-hbmxt"
Oct 30 21:42:24.496624 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:24.496537    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f152360c-cf62-4d10-b2e5-b85ac7dfb739-metrics-tls\") pod \"dns-default-hbmxt\" (UID: \"f152360c-cf62-4d10-b2e5-b85ac7dfb739\") " pod="openshift-dns/dns-default-hbmxt"
Oct 30 21:42:24.496624 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:24.496595    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n6qcb\" (UniqueName: \"kubernetes.io/projected/f152360c-cf62-4d10-b2e5-b85ac7dfb739-kube-api-access-n6qcb\") pod \"dns-default-hbmxt\" (UID: \"f152360c-cf62-4d10-b2e5-b85ac7dfb739\") " pod="openshift-dns/dns-default-hbmxt"
Oct 30 21:42:24.496722 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:24.496628    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/f152360c-cf62-4d10-b2e5-b85ac7dfb739-config-volume\") pod \"dns-default-hbmxt\" (UID: \"f152360c-cf62-4d10-b2e5-b85ac7dfb739\") " pod="openshift-dns/dns-default-hbmxt"
Oct 30 21:42:24.496722 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:24.496656    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7vks4\" (UniqueName: \"kubernetes.io/projected/4436ec50-e190-4c1c-8b6c-2b9b5d12cb9f-kube-api-access-7vks4\") pod \"ingress-canary-5m6cr\" (UID: \"4436ec50-e190-4c1c-8b6c-2b9b5d12cb9f\") " pod="openshift-ingress-canary/ingress-canary-5m6cr"
Oct 30 21:42:24.598125 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:24.598039    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7vks4\" (UniqueName: \"kubernetes.io/projected/4436ec50-e190-4c1c-8b6c-2b9b5d12cb9f-kube-api-access-7vks4\") pod \"ingress-canary-5m6cr\" (UID: \"4436ec50-e190-4c1c-8b6c-2b9b5d12cb9f\") " pod="openshift-ingress-canary/ingress-canary-5m6cr"
Oct 30 21:42:24.598125 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:24.598094    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4436ec50-e190-4c1c-8b6c-2b9b5d12cb9f-cert\") pod \"ingress-canary-5m6cr\" (UID: \"4436ec50-e190-4c1c-8b6c-2b9b5d12cb9f\") " pod="openshift-ingress-canary/ingress-canary-5m6cr"
Oct 30 21:42:24.598125 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:24.598112    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f152360c-cf62-4d10-b2e5-b85ac7dfb739-tmp-dir\") pod \"dns-default-hbmxt\" (UID: \"f152360c-cf62-4d10-b2e5-b85ac7dfb739\") " pod="openshift-dns/dns-default-hbmxt"
Oct 30 21:42:24.598394 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:24.598135    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f152360c-cf62-4d10-b2e5-b85ac7dfb739-metrics-tls\") pod \"dns-default-hbmxt\" (UID: \"f152360c-cf62-4d10-b2e5-b85ac7dfb739\") " pod="openshift-dns/dns-default-hbmxt"
Oct 30 21:42:24.598394 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:24.598160    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n6qcb\" (UniqueName: \"kubernetes.io/projected/f152360c-cf62-4d10-b2e5-b85ac7dfb739-kube-api-access-n6qcb\") pod \"dns-default-hbmxt\" (UID: \"f152360c-cf62-4d10-b2e5-b85ac7dfb739\") " pod="openshift-dns/dns-default-hbmxt"
Oct 30 21:42:24.598394 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:24.598176    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/f152360c-cf62-4d10-b2e5-b85ac7dfb739-config-volume\") pod \"dns-default-hbmxt\" (UID: \"f152360c-cf62-4d10-b2e5-b85ac7dfb739\") " pod="openshift-dns/dns-default-hbmxt"
Oct 30 21:42:24.598394 ip-10-0-140-18 kubenswrapper[2580]: E1030 21:42:24.598292    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 30 21:42:24.598394 ip-10-0-140-18 kubenswrapper[2580]: E1030 21:42:24.598374    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f152360c-cf62-4d10-b2e5-b85ac7dfb739-metrics-tls podName:f152360c-cf62-4d10-b2e5-b85ac7dfb739 nodeName:}" failed. No retries permitted until 2026-10-30 21:42:25.098356937 +0000 UTC m=+32.212299256 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f152360c-cf62-4d10-b2e5-b85ac7dfb739-metrics-tls") pod "dns-default-hbmxt" (UID: "f152360c-cf62-4d10-b2e5-b85ac7dfb739") : secret "dns-default-metrics-tls" not found
Oct 30 21:42:24.598612 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:24.598573    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f152360c-cf62-4d10-b2e5-b85ac7dfb739-tmp-dir\") pod \"dns-default-hbmxt\" (UID: \"f152360c-cf62-4d10-b2e5-b85ac7dfb739\") " pod="openshift-dns/dns-default-hbmxt"
Oct 30 21:42:24.598794 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:24.598776    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/f152360c-cf62-4d10-b2e5-b85ac7dfb739-config-volume\") pod \"dns-default-hbmxt\" (UID: \"f152360c-cf62-4d10-b2e5-b85ac7dfb739\") " pod="openshift-dns/dns-default-hbmxt"
Oct 30 21:42:24.601876 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:24.601850    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4436ec50-e190-4c1c-8b6c-2b9b5d12cb9f-cert\") pod \"ingress-canary-5m6cr\" (UID: \"4436ec50-e190-4c1c-8b6c-2b9b5d12cb9f\") " pod="openshift-ingress-canary/ingress-canary-5m6cr"
Oct 30 21:42:24.660534 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:24.660496    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7vks4\" (UniqueName: \"kubernetes.io/projected/4436ec50-e190-4c1c-8b6c-2b9b5d12cb9f-kube-api-access-7vks4\") pod \"ingress-canary-5m6cr\" (UID: \"4436ec50-e190-4c1c-8b6c-2b9b5d12cb9f\") " pod="openshift-ingress-canary/ingress-canary-5m6cr"
Oct 30 21:42:24.660668 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:24.660496    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n6qcb\" (UniqueName: \"kubernetes.io/projected/f152360c-cf62-4d10-b2e5-b85ac7dfb739-kube-api-access-n6qcb\") pod \"dns-default-hbmxt\" (UID: \"f152360c-cf62-4d10-b2e5-b85ac7dfb739\") " pod="openshift-dns/dns-default-hbmxt"
Oct 30 21:42:24.688601 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:24.688570    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-5m6cr"
Oct 30 21:42:25.066338 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:25.066309    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-5m6cr"]
Oct 30 21:42:25.069675 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:42:25.069648    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4436ec50_e190_4c1c_8b6c_2b9b5d12cb9f.slice/crio-7a14b2934bbe1c4ee21f714f16eb03e32018d69985a144c98a7450961a811688 WatchSource:0}: Error finding container 7a14b2934bbe1c4ee21f714f16eb03e32018d69985a144c98a7450961a811688: Status 404 returned error can't find the container with id 7a14b2934bbe1c4ee21f714f16eb03e32018d69985a144c98a7450961a811688
Oct 30 21:42:25.101455 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:25.101427    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f152360c-cf62-4d10-b2e5-b85ac7dfb739-metrics-tls\") pod \"dns-default-hbmxt\" (UID: \"f152360c-cf62-4d10-b2e5-b85ac7dfb739\") " pod="openshift-dns/dns-default-hbmxt"
Oct 30 21:42:25.101554 ip-10-0-140-18 kubenswrapper[2580]: E1030 21:42:25.101546    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 30 21:42:25.101606 ip-10-0-140-18 kubenswrapper[2580]: E1030 21:42:25.101593    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f152360c-cf62-4d10-b2e5-b85ac7dfb739-metrics-tls podName:f152360c-cf62-4d10-b2e5-b85ac7dfb739 nodeName:}" failed. No retries permitted until 2026-10-30 21:42:26.101581459 +0000 UTC m=+33.215523762 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f152360c-cf62-4d10-b2e5-b85ac7dfb739-metrics-tls") pod "dns-default-hbmxt" (UID: "f152360c-cf62-4d10-b2e5-b85ac7dfb739") : secret "dns-default-metrics-tls" not found
Oct 30 21:42:25.375337 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:25.375297    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dt7cq"
Oct 30 21:42:25.375760 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:25.375542    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9st74"
Oct 30 21:42:25.427763 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:25.427727    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Oct 30 21:42:25.427763 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:25.427728    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Oct 30 21:42:25.428014 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:25.427818    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-dcz6k\""
Oct 30 21:42:25.428014 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:25.427834    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Oct 30 21:42:25.428014 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:25.427850    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-msl6j\""
Oct 30 21:42:25.558519 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:25.558483    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-5m6cr" event={"ID":"4436ec50-e190-4c1c-8b6c-2b9b5d12cb9f","Type":"ContainerStarted","Data":"7a14b2934bbe1c4ee21f714f16eb03e32018d69985a144c98a7450961a811688"}
Oct 30 21:42:26.109195 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:26.109154    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f152360c-cf62-4d10-b2e5-b85ac7dfb739-metrics-tls\") pod \"dns-default-hbmxt\" (UID: \"f152360c-cf62-4d10-b2e5-b85ac7dfb739\") " pod="openshift-dns/dns-default-hbmxt"
Oct 30 21:42:26.109356 ip-10-0-140-18 kubenswrapper[2580]: E1030 21:42:26.109282    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 30 21:42:26.109356 ip-10-0-140-18 kubenswrapper[2580]: E1030 21:42:26.109333    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f152360c-cf62-4d10-b2e5-b85ac7dfb739-metrics-tls podName:f152360c-cf62-4d10-b2e5-b85ac7dfb739 nodeName:}" failed. No retries permitted until 2026-10-30 21:42:28.109321217 +0000 UTC m=+35.223263515 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f152360c-cf62-4d10-b2e5-b85ac7dfb739-metrics-tls") pod "dns-default-hbmxt" (UID: "f152360c-cf62-4d10-b2e5-b85ac7dfb739") : secret "dns-default-metrics-tls" not found
Oct 30 21:42:27.036078 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:27.036049    2580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-fncvg"]
Oct 30 21:42:27.042660 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:27.042632    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-fncvg"
Oct 30 21:42:27.118162 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:27.118128    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ccc5af13-906a-469d-9ca5-09366d1a7e93-metrics-certs\") pod \"network-metrics-daemon-8jvqh\" (UID: \"ccc5af13-906a-469d-9ca5-09366d1a7e93\") " pod="openshift-multus/network-metrics-daemon-8jvqh"
Oct 30 21:42:27.118343 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:27.118190    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0032cb99-3a37-40c6-98cb-2d40ae4a9319-original-pull-secret\") pod \"global-pull-secret-syncer-dt7cq\" (UID: \"0032cb99-3a37-40c6-98cb-2d40ae4a9319\") " pod="kube-system/global-pull-secret-syncer-dt7cq"
Oct 30 21:42:27.118406 ip-10-0-140-18 kubenswrapper[2580]: E1030 21:42:27.118348    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Oct 30 21:42:27.118478 ip-10-0-140-18 kubenswrapper[2580]: E1030 21:42:27.118432    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ccc5af13-906a-469d-9ca5-09366d1a7e93-metrics-certs podName:ccc5af13-906a-469d-9ca5-09366d1a7e93 nodeName:}" failed. No retries permitted until 2026-10-30 21:42:59.118403555 +0000 UTC m=+66.232345857 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ccc5af13-906a-469d-9ca5-09366d1a7e93-metrics-certs") pod "network-metrics-daemon-8jvqh" (UID: "ccc5af13-906a-469d-9ca5-09366d1a7e93") : secret "metrics-daemon-secret" not found
Oct 30 21:42:27.121185 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:27.121143    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0032cb99-3a37-40c6-98cb-2d40ae4a9319-original-pull-secret\") pod \"global-pull-secret-syncer-dt7cq\" (UID: \"0032cb99-3a37-40c6-98cb-2d40ae4a9319\") " pod="kube-system/global-pull-secret-syncer-dt7cq"
Oct 30 21:42:27.154610 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:27.154576    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Oct 30 21:42:27.154610 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:27.154601    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Oct 30 21:42:27.154838 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:27.154576    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Oct 30 21:42:27.154838 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:27.154676    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Oct 30 21:42:27.155246 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:27.155212    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-gbz5v\""
Oct 30 21:42:27.157100 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:27.157078    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-fncvg"]
Oct 30 21:42:27.188338 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:27.188304    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dt7cq"
Oct 30 21:42:27.218799 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:27.218765    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/32a1c660-effa-4b6c-975a-dbb68c468eda-crio-socket\") pod \"insights-runtime-extractor-fncvg\" (UID: \"32a1c660-effa-4b6c-975a-dbb68c468eda\") " pod="openshift-insights/insights-runtime-extractor-fncvg"
Oct 30 21:42:27.218990 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:27.218822    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zzqrt\" (UniqueName: \"kubernetes.io/projected/32a1c660-effa-4b6c-975a-dbb68c468eda-kube-api-access-zzqrt\") pod \"insights-runtime-extractor-fncvg\" (UID: \"32a1c660-effa-4b6c-975a-dbb68c468eda\") " pod="openshift-insights/insights-runtime-extractor-fncvg"
Oct 30 21:42:27.218990 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:27.218859    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/32a1c660-effa-4b6c-975a-dbb68c468eda-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fncvg\" (UID: \"32a1c660-effa-4b6c-975a-dbb68c468eda\") " pod="openshift-insights/insights-runtime-extractor-fncvg"
Oct 30 21:42:27.218990 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:27.218960    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/32a1c660-effa-4b6c-975a-dbb68c468eda-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fncvg\" (UID: \"32a1c660-effa-4b6c-975a-dbb68c468eda\") " pod="openshift-insights/insights-runtime-extractor-fncvg"
Oct 30 21:42:27.218990 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:27.218983    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/32a1c660-effa-4b6c-975a-dbb68c468eda-data-volume\") pod \"insights-runtime-extractor-fncvg\" (UID: \"32a1c660-effa-4b6c-975a-dbb68c468eda\") " pod="openshift-insights/insights-runtime-extractor-fncvg"
Oct 30 21:42:27.230637 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:27.230604    2580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-gj8rz"]
Oct 30 21:42:27.235804 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:27.235780    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-gj8rz"
Oct 30 21:42:27.267178 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:27.266408    2580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/periodic-gathering-bcmkf-fm7wg"]
Oct 30 21:42:27.269238 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:27.269214    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-bcmkf-fm7wg"
Oct 30 21:42:27.296175 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:27.296107    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-gj8rz"]
Oct 30 21:42:27.310503 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:27.310476    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Oct 30 21:42:27.310503 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:27.310493    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-5cjr4\""
Oct 30 21:42:27.310503 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:27.310499    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Oct 30 21:42:27.319965 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:27.319910    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/32a1c660-effa-4b6c-975a-dbb68c468eda-crio-socket\") pod \"insights-runtime-extractor-fncvg\" (UID: \"32a1c660-effa-4b6c-975a-dbb68c468eda\") " pod="openshift-insights/insights-runtime-extractor-fncvg"
Oct 30 21:42:27.320107 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:27.319977    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zzqrt\" (UniqueName: \"kubernetes.io/projected/32a1c660-effa-4b6c-975a-dbb68c468eda-kube-api-access-zzqrt\") pod \"insights-runtime-extractor-fncvg\" (UID: \"32a1c660-effa-4b6c-975a-dbb68c468eda\") " pod="openshift-insights/insights-runtime-extractor-fncvg"
Oct 30 21:42:27.320107 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:27.320008    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/32a1c660-effa-4b6c-975a-dbb68c468eda-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fncvg\" (UID: \"32a1c660-effa-4b6c-975a-dbb68c468eda\") " pod="openshift-insights/insights-runtime-extractor-fncvg"
Oct 30 21:42:27.320107 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:27.320046    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-js6zb\" (UniqueName: \"kubernetes.io/projected/10d2e5ae-c1c4-400c-8c67-3aebb6caaa6a-kube-api-access-js6zb\") pod \"network-check-target-9st74\" (UID: \"10d2e5ae-c1c4-400c-8c67-3aebb6caaa6a\") " pod="openshift-network-diagnostics/network-check-target-9st74"
Oct 30 21:42:27.320107 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:27.320091    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/32a1c660-effa-4b6c-975a-dbb68c468eda-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fncvg\" (UID: \"32a1c660-effa-4b6c-975a-dbb68c468eda\") " pod="openshift-insights/insights-runtime-extractor-fncvg"
Oct 30 21:42:27.320107 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:27.320109    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/32a1c660-effa-4b6c-975a-dbb68c468eda-data-volume\") pod \"insights-runtime-extractor-fncvg\" (UID: \"32a1c660-effa-4b6c-975a-dbb68c468eda\") " pod="openshift-insights/insights-runtime-extractor-fncvg"
Oct 30 21:42:27.320428 ip-10-0-140-18 kubenswrapper[2580]: E1030 21:42:27.320121    2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Oct 30 21:42:27.320428 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:27.320160    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/32a1c660-effa-4b6c-975a-dbb68c468eda-crio-socket\") pod \"insights-runtime-extractor-fncvg\" (UID: \"32a1c660-effa-4b6c-975a-dbb68c468eda\") " pod="openshift-insights/insights-runtime-extractor-fncvg"
Oct 30 21:42:27.320428 ip-10-0-140-18 kubenswrapper[2580]: E1030 21:42:27.320189    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/32a1c660-effa-4b6c-975a-dbb68c468eda-kube-rbac-proxy-cm podName:32a1c660-effa-4b6c-975a-dbb68c468eda nodeName:}" failed. No retries permitted until 2026-10-30 21:42:27.820170569 +0000 UTC m=+34.934112901 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/32a1c660-effa-4b6c-975a-dbb68c468eda-kube-rbac-proxy-cm") pod "insights-runtime-extractor-fncvg" (UID: "32a1c660-effa-4b6c-975a-dbb68c468eda") : configmap "kube-rbac-proxy" not found
Oct 30 21:42:27.320428 ip-10-0-140-18 kubenswrapper[2580]: E1030 21:42:27.320284    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 30 21:42:27.320428 ip-10-0-140-18 kubenswrapper[2580]: E1030 21:42:27.320340    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/32a1c660-effa-4b6c-975a-dbb68c468eda-insights-runtime-extractor-tls podName:32a1c660-effa-4b6c-975a-dbb68c468eda nodeName:}" failed. No retries permitted until 2026-10-30 21:42:27.82032215 +0000 UTC m=+34.934264446 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/32a1c660-effa-4b6c-975a-dbb68c468eda-insights-runtime-extractor-tls") pod "insights-runtime-extractor-fncvg" (UID: "32a1c660-effa-4b6c-975a-dbb68c468eda") : secret "insights-runtime-extractor-tls" not found
Oct 30 21:42:27.320764 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:27.320523    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/32a1c660-effa-4b6c-975a-dbb68c468eda-data-volume\") pod \"insights-runtime-extractor-fncvg\" (UID: \"32a1c660-effa-4b6c-975a-dbb68c468eda\") " pod="openshift-insights/insights-runtime-extractor-fncvg"
Oct 30 21:42:27.321093 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:27.320782    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Oct 30 21:42:27.321439 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:27.321422    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-9lfqr\""
Oct 30 21:42:27.323358 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:27.323337    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-js6zb\" (UniqueName: \"kubernetes.io/projected/10d2e5ae-c1c4-400c-8c67-3aebb6caaa6a-kube-api-access-js6zb\") pod \"network-check-target-9st74\" (UID: \"10d2e5ae-c1c4-400c-8c67-3aebb6caaa6a\") " pod="openshift-network-diagnostics/network-check-target-9st74"
Oct 30 21:42:27.394224 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:27.394188    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Oct 30 21:42:27.410653 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:27.410621    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/periodic-gathering-bcmkf-fm7wg"]
Oct 30 21:42:27.412352 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:27.412328    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zzqrt\" (UniqueName: \"kubernetes.io/projected/32a1c660-effa-4b6c-975a-dbb68c468eda-kube-api-access-zzqrt\") pod \"insights-runtime-extractor-fncvg\" (UID: \"32a1c660-effa-4b6c-975a-dbb68c468eda\") " pod="openshift-insights/insights-runtime-extractor-fncvg"
Oct 30 21:42:27.421344 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:27.421321    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bddede0e-ee07-4344-878d-10b967435e13-service-ca-bundle\") pod \"periodic-gathering-bcmkf-fm7wg\" (UID: \"bddede0e-ee07-4344-878d-10b967435e13\") " pod="openshift-insights/periodic-gathering-bcmkf-fm7wg"
Oct 30 21:42:27.421453 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:27.421360    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lnn4t\" (UniqueName: \"kubernetes.io/projected/bddede0e-ee07-4344-878d-10b967435e13-kube-api-access-lnn4t\") pod \"periodic-gathering-bcmkf-fm7wg\" (UID: \"bddede0e-ee07-4344-878d-10b967435e13\") " pod="openshift-insights/periodic-gathering-bcmkf-fm7wg"
Oct 30 21:42:27.421510 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:27.421461    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/bddede0e-ee07-4344-878d-10b967435e13-archives-path\") pod \"periodic-gathering-bcmkf-fm7wg\" (UID: \"bddede0e-ee07-4344-878d-10b967435e13\") " pod="openshift-insights/periodic-gathering-bcmkf-fm7wg"
Oct 30 21:42:27.421510 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:27.421485    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bddede0e-ee07-4344-878d-10b967435e13-trusted-ca-bundle\") pod \"periodic-gathering-bcmkf-fm7wg\" (UID: \"bddede0e-ee07-4344-878d-10b967435e13\") " pod="openshift-insights/periodic-gathering-bcmkf-fm7wg"
Oct 30 21:42:27.421510 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:27.421503    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g98k9\" (UniqueName: \"kubernetes.io/projected/525a0437-9a44-425b-966a-b118d91a4522-kube-api-access-g98k9\") pod \"migrator-66df5d496b-gj8rz\" (UID: \"525a0437-9a44-425b-966a-b118d91a4522\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-gj8rz"
Oct 30 21:42:27.491901 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:27.491859    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9st74"
Oct 30 21:42:27.522211 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:27.522183    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/bddede0e-ee07-4344-878d-10b967435e13-archives-path\") pod \"periodic-gathering-bcmkf-fm7wg\" (UID: \"bddede0e-ee07-4344-878d-10b967435e13\") " pod="openshift-insights/periodic-gathering-bcmkf-fm7wg"
Oct 30 21:42:27.522655 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:27.522221    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bddede0e-ee07-4344-878d-10b967435e13-trusted-ca-bundle\") pod \"periodic-gathering-bcmkf-fm7wg\" (UID: \"bddede0e-ee07-4344-878d-10b967435e13\") " pod="openshift-insights/periodic-gathering-bcmkf-fm7wg"
Oct 30 21:42:27.522655 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:27.522271    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g98k9\" (UniqueName: \"kubernetes.io/projected/525a0437-9a44-425b-966a-b118d91a4522-kube-api-access-g98k9\") pod \"migrator-66df5d496b-gj8rz\" (UID: \"525a0437-9a44-425b-966a-b118d91a4522\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-gj8rz"
Oct 30 21:42:27.522655 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:27.522354    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bddede0e-ee07-4344-878d-10b967435e13-service-ca-bundle\") pod \"periodic-gathering-bcmkf-fm7wg\" (UID: \"bddede0e-ee07-4344-878d-10b967435e13\") " pod="openshift-insights/periodic-gathering-bcmkf-fm7wg"
Oct 30 21:42:27.522655 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:27.522383    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lnn4t\" (UniqueName: \"kubernetes.io/projected/bddede0e-ee07-4344-878d-10b967435e13-kube-api-access-lnn4t\") pod \"periodic-gathering-bcmkf-fm7wg\" (UID: \"bddede0e-ee07-4344-878d-10b967435e13\") " pod="openshift-insights/periodic-gathering-bcmkf-fm7wg"
Oct 30 21:42:27.522886 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:27.522715    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/bddede0e-ee07-4344-878d-10b967435e13-archives-path\") pod \"periodic-gathering-bcmkf-fm7wg\" (UID: \"bddede0e-ee07-4344-878d-10b967435e13\") " pod="openshift-insights/periodic-gathering-bcmkf-fm7wg"
Oct 30 21:42:27.523473 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:27.523445    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bddede0e-ee07-4344-878d-10b967435e13-service-ca-bundle\") pod \"periodic-gathering-bcmkf-fm7wg\" (UID: \"bddede0e-ee07-4344-878d-10b967435e13\") " pod="openshift-insights/periodic-gathering-bcmkf-fm7wg"
Oct 30 21:42:27.523748 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:27.523721    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bddede0e-ee07-4344-878d-10b967435e13-trusted-ca-bundle\") pod \"periodic-gathering-bcmkf-fm7wg\" (UID: \"bddede0e-ee07-4344-878d-10b967435e13\") " pod="openshift-insights/periodic-gathering-bcmkf-fm7wg"
Oct 30 21:42:27.566572 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:27.566499    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-dt7cq"]
Oct 30 21:42:27.569484 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:42:27.569448    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0032cb99_3a37_40c6_98cb_2d40ae4a9319.slice/crio-5e9d9d5b697e87a3519595619e275bc9ad494e247a8e7e1892ea4d63b50a4a56 WatchSource:0}: Error finding container 5e9d9d5b697e87a3519595619e275bc9ad494e247a8e7e1892ea4d63b50a4a56: Status 404 returned error can't find the container with id 5e9d9d5b697e87a3519595619e275bc9ad494e247a8e7e1892ea4d63b50a4a56
Oct 30 21:42:27.582182 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:27.582161    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g98k9\" (UniqueName: \"kubernetes.io/projected/525a0437-9a44-425b-966a-b118d91a4522-kube-api-access-g98k9\") pod \"migrator-66df5d496b-gj8rz\" (UID: \"525a0437-9a44-425b-966a-b118d91a4522\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-gj8rz"
Oct 30 21:42:27.584241 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:27.584218    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lnn4t\" (UniqueName: \"kubernetes.io/projected/bddede0e-ee07-4344-878d-10b967435e13-kube-api-access-lnn4t\") pod \"periodic-gathering-bcmkf-fm7wg\" (UID: \"bddede0e-ee07-4344-878d-10b967435e13\") " pod="openshift-insights/periodic-gathering-bcmkf-fm7wg"
Oct 30 21:42:27.825761 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:27.825680    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/32a1c660-effa-4b6c-975a-dbb68c468eda-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fncvg\" (UID: \"32a1c660-effa-4b6c-975a-dbb68c468eda\") " pod="openshift-insights/insights-runtime-extractor-fncvg"
Oct 30 21:42:27.825908 ip-10-0-140-18 kubenswrapper[2580]: E1030 21:42:27.825871    2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Oct 30 21:42:27.825988 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:27.825911    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/32a1c660-effa-4b6c-975a-dbb68c468eda-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fncvg\" (UID: \"32a1c660-effa-4b6c-975a-dbb68c468eda\") " pod="openshift-insights/insights-runtime-extractor-fncvg"
Oct 30 21:42:27.826040 ip-10-0-140-18 kubenswrapper[2580]: E1030 21:42:27.825991    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/32a1c660-effa-4b6c-975a-dbb68c468eda-kube-rbac-proxy-cm podName:32a1c660-effa-4b6c-975a-dbb68c468eda nodeName:}" failed. No retries permitted until 2026-10-30 21:42:28.825970562 +0000 UTC m=+35.939912859 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/32a1c660-effa-4b6c-975a-dbb68c468eda-kube-rbac-proxy-cm") pod "insights-runtime-extractor-fncvg" (UID: "32a1c660-effa-4b6c-975a-dbb68c468eda") : configmap "kube-rbac-proxy" not found
Oct 30 21:42:27.826154 ip-10-0-140-18 kubenswrapper[2580]: E1030 21:42:27.826132    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 30 21:42:27.826253 ip-10-0-140-18 kubenswrapper[2580]: E1030 21:42:27.826236    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/32a1c660-effa-4b6c-975a-dbb68c468eda-insights-runtime-extractor-tls podName:32a1c660-effa-4b6c-975a-dbb68c468eda nodeName:}" failed. No retries permitted until 2026-10-30 21:42:28.826190653 +0000 UTC m=+35.940132953 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/32a1c660-effa-4b6c-975a-dbb68c468eda-insights-runtime-extractor-tls") pod "insights-runtime-extractor-fncvg" (UID: "32a1c660-effa-4b6c-975a-dbb68c468eda") : secret "insights-runtime-extractor-tls" not found
Oct 30 21:42:27.850894 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:27.850859    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-gj8rz"
Oct 30 21:42:27.883143 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:27.883113    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-bcmkf-fm7wg"
Oct 30 21:42:27.892605 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:27.892571    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-9st74"]
Oct 30 21:42:27.895508 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:42:27.895472    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10d2e5ae_c1c4_400c_8c67_3aebb6caaa6a.slice/crio-c30e39faca61472a40248b9cb2c2aecc5cd89074cddd409ced6aa73a7e97ff49 WatchSource:0}: Error finding container c30e39faca61472a40248b9cb2c2aecc5cd89074cddd409ced6aa73a7e97ff49: Status 404 returned error can't find the container with id c30e39faca61472a40248b9cb2c2aecc5cd89074cddd409ced6aa73a7e97ff49
Oct 30 21:42:28.130251 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:28.130212    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f152360c-cf62-4d10-b2e5-b85ac7dfb739-metrics-tls\") pod \"dns-default-hbmxt\" (UID: \"f152360c-cf62-4d10-b2e5-b85ac7dfb739\") " pod="openshift-dns/dns-default-hbmxt"
Oct 30 21:42:28.130706 ip-10-0-140-18 kubenswrapper[2580]: E1030 21:42:28.130404    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 30 21:42:28.130706 ip-10-0-140-18 kubenswrapper[2580]: E1030 21:42:28.130485    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f152360c-cf62-4d10-b2e5-b85ac7dfb739-metrics-tls podName:f152360c-cf62-4d10-b2e5-b85ac7dfb739 nodeName:}" failed. No retries permitted until 2026-10-30 21:42:32.130465496 +0000 UTC m=+39.244407790 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f152360c-cf62-4d10-b2e5-b85ac7dfb739-metrics-tls") pod "dns-default-hbmxt" (UID: "f152360c-cf62-4d10-b2e5-b85ac7dfb739") : secret "dns-default-metrics-tls" not found
Oct 30 21:42:28.254575 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:28.254506    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-gj8rz"]
Oct 30 21:42:28.260037 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:42:28.260009    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod525a0437_9a44_425b_966a_b118d91a4522.slice/crio-ea9ef8f120528befa45af4e9c5d895ac4de2f82aa8e6d58db49358916aa15775 WatchSource:0}: Error finding container ea9ef8f120528befa45af4e9c5d895ac4de2f82aa8e6d58db49358916aa15775: Status 404 returned error can't find the container with id ea9ef8f120528befa45af4e9c5d895ac4de2f82aa8e6d58db49358916aa15775
Oct 30 21:42:28.301790 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:28.301769    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/periodic-gathering-bcmkf-fm7wg"]
Oct 30 21:42:28.304732 ip-10-0-140-18 kubenswrapper[2580]: W1030 21:42:28.304699    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbddede0e_ee07_4344_878d_10b967435e13.slice/crio-76411d00e5efeac0f8e2cdc0c51751cb906f91e91e8b427c7cd78c85f25ebfa6 WatchSource:0}: Error finding container 76411d00e5efeac0f8e2cdc0c51751cb906f91e91e8b427c7cd78c85f25ebfa6: Status 404 returned error can't find the container with id 76411d00e5efeac0f8e2cdc0c51751cb906f91e91e8b427c7cd78c85f25ebfa6
Oct 30 21:42:28.568644 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:28.568570    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-gj8rz" event={"ID":"525a0437-9a44-425b-966a-b118d91a4522","Type":"ContainerStarted","Data":"ea9ef8f120528befa45af4e9c5d895ac4de2f82aa8e6d58db49358916aa15775"}
Oct 30 21:42:28.570327 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:28.570294    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-dt7cq" event={"ID":"0032cb99-3a37-40c6-98cb-2d40ae4a9319","Type":"ContainerStarted","Data":"5e9d9d5b697e87a3519595619e275bc9ad494e247a8e7e1892ea4d63b50a4a56"}
Oct 30 21:42:28.571630 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:28.571602    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-bcmkf-fm7wg" event={"ID":"bddede0e-ee07-4344-878d-10b967435e13","Type":"ContainerStarted","Data":"76411d00e5efeac0f8e2cdc0c51751cb906f91e91e8b427c7cd78c85f25ebfa6"}
Oct 30 21:42:28.573217 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:28.573191    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-9st74" event={"ID":"10d2e5ae-c1c4-400c-8c67-3aebb6caaa6a","Type":"ContainerStarted","Data":"c30e39faca61472a40248b9cb2c2aecc5cd89074cddd409ced6aa73a7e97ff49"}
Oct 30 21:42:28.836371 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:28.836294    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/32a1c660-effa-4b6c-975a-dbb68c468eda-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fncvg\" (UID: \"32a1c660-effa-4b6c-975a-dbb68c468eda\") " pod="openshift-insights/insights-runtime-extractor-fncvg"
Oct 30 21:42:28.836517 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:28.836373    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/32a1c660-effa-4b6c-975a-dbb68c468eda-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fncvg\" (UID: \"32a1c660-effa-4b6c-975a-dbb68c468eda\") " pod="openshift-insights/insights-runtime-extractor-fncvg"
Oct 30 21:42:28.836517 ip-10-0-140-18 kubenswrapper[2580]: E1030 21:42:28.836504    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 30 21:42:28.836588 ip-10-0-140-18 kubenswrapper[2580]: E1030 21:42:28.836560    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/32a1c660-effa-4b6c-975a-dbb68c468eda-insights-runtime-extractor-tls podName:32a1c660-effa-4b6c-975a-dbb68c468eda nodeName:}" failed. No retries permitted until 2026-10-30 21:42:30.836544049 +0000 UTC m=+37.950486354 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/32a1c660-effa-4b6c-975a-dbb68c468eda-insights-runtime-extractor-tls") pod "insights-runtime-extractor-fncvg" (UID: "32a1c660-effa-4b6c-975a-dbb68c468eda") : secret "insights-runtime-extractor-tls" not found
Oct 30 21:42:28.836981 ip-10-0-140-18 kubenswrapper[2580]: E1030 21:42:28.836962    2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Oct 30 21:42:28.837068 ip-10-0-140-18 kubenswrapper[2580]: E1030 21:42:28.837019    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/32a1c660-effa-4b6c-975a-dbb68c468eda-kube-rbac-proxy-cm podName:32a1c660-effa-4b6c-975a-dbb68c468eda nodeName:}" failed. No retries permitted until 2026-10-30 21:42:30.837003431 +0000 UTC m=+37.950945728 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/32a1c660-effa-4b6c-975a-dbb68c468eda-kube-rbac-proxy-cm") pod "insights-runtime-extractor-fncvg" (UID: "32a1c660-effa-4b6c-975a-dbb68c468eda") : configmap "kube-rbac-proxy" not found
Oct 30 21:42:29.580800 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:29.580760    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-5m6cr" event={"ID":"4436ec50-e190-4c1c-8b6c-2b9b5d12cb9f","Type":"ContainerStarted","Data":"830f2c77888306aa2ed289931db5943541e723a19d7b5ec7d49397cd85315be1"}
Oct 30 21:42:30.588048 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:30.587994    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-gj8rz" event={"ID":"525a0437-9a44-425b-966a-b118d91a4522","Type":"ContainerStarted","Data":"2035e6294f844d52991f6231d77b918cc50c8f41d5fcf1e8a6f22f93893f8f01"}
Oct 30 21:42:30.702429 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:30.702333    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-5m6cr" podStartSLOduration=2.814953739 podStartE2EDuration="6.70231695s" totalImagesPullingTime="3.887363211s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:42:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:42:29.694945005 +0000 UTC m=+36.808887315" watchObservedRunningTime="2026-10-30 21:42:30.70231695 +0000 UTC m=+37.816259263"
Oct 30 21:42:30.702429 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:30.702416    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-gj8rz" podStartSLOduration=1.524143923 podStartE2EDuration="3.70241191s" totalImagesPullingTime="2.178267987s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:42:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:42:30.701060826 +0000 UTC m=+37.815003140" watchObservedRunningTime="2026-10-30 21:42:30.70241191 +0000 UTC m=+37.816354223"
Oct 30 21:42:30.853411 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:30.853373    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/32a1c660-effa-4b6c-975a-dbb68c468eda-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fncvg\" (UID: \"32a1c660-effa-4b6c-975a-dbb68c468eda\") " pod="openshift-insights/insights-runtime-extractor-fncvg"
Oct 30 21:42:30.853578 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:30.853459    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/32a1c660-effa-4b6c-975a-dbb68c468eda-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fncvg\" (UID: \"32a1c660-effa-4b6c-975a-dbb68c468eda\") " pod="openshift-insights/insights-runtime-extractor-fncvg"
Oct 30 21:42:30.853578 ip-10-0-140-18 kubenswrapper[2580]: E1030 21:42:30.853502    2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Oct 30 21:42:30.853578 ip-10-0-140-18 kubenswrapper[2580]: E1030 21:42:30.853567    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 30 21:42:30.853738 ip-10-0-140-18 kubenswrapper[2580]: E1030 21:42:30.853603    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/32a1c660-effa-4b6c-975a-dbb68c468eda-kube-rbac-proxy-cm podName:32a1c660-effa-4b6c-975a-dbb68c468eda nodeName:}" failed. No retries permitted until 2026-10-30 21:42:34.853550569 +0000 UTC m=+41.967492875 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/32a1c660-effa-4b6c-975a-dbb68c468eda-kube-rbac-proxy-cm") pod "insights-runtime-extractor-fncvg" (UID: "32a1c660-effa-4b6c-975a-dbb68c468eda") : configmap "kube-rbac-proxy" not found
Oct 30 21:42:30.853738 ip-10-0-140-18 kubenswrapper[2580]: E1030 21:42:30.853661    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/32a1c660-effa-4b6c-975a-dbb68c468eda-insights-runtime-extractor-tls podName:32a1c660-effa-4b6c-975a-dbb68c468eda nodeName:}" failed. No retries permitted until 2026-10-30 21:42:34.853649823 +0000 UTC m=+41.967592130 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/32a1c660-effa-4b6c-975a-dbb68c468eda-insights-runtime-extractor-tls") pod "insights-runtime-extractor-fncvg" (UID: "32a1c660-effa-4b6c-975a-dbb68c468eda") : secret "insights-runtime-extractor-tls" not found
Oct 30 21:42:32.163094 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:32.163061    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f152360c-cf62-4d10-b2e5-b85ac7dfb739-metrics-tls\") pod \"dns-default-hbmxt\" (UID: \"f152360c-cf62-4d10-b2e5-b85ac7dfb739\") " pod="openshift-dns/dns-default-hbmxt"
Oct 30 21:42:32.163571 ip-10-0-140-18 kubenswrapper[2580]: E1030 21:42:32.163206    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 30 21:42:32.163571 ip-10-0-140-18 kubenswrapper[2580]: E1030 21:42:32.163270    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f152360c-cf62-4d10-b2e5-b85ac7dfb739-metrics-tls podName:f152360c-cf62-4d10-b2e5-b85ac7dfb739 nodeName:}" failed. No retries permitted until 2026-10-30 21:42:40.163255641 +0000 UTC m=+47.277197937 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f152360c-cf62-4d10-b2e5-b85ac7dfb739-metrics-tls") pod "dns-default-hbmxt" (UID: "f152360c-cf62-4d10-b2e5-b85ac7dfb739") : secret "dns-default-metrics-tls" not found
Oct 30 21:42:32.594457 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:32.594412    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-bcmkf-fm7wg" event={"ID":"bddede0e-ee07-4344-878d-10b967435e13","Type":"ContainerStarted","Data":"652149a047dc3500bf129446aecba9d8215abcd5ed7fecbe2d568df3d202f4c5"}
Oct 30 21:42:32.710327 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:32.710269    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-insights/periodic-gathering-bcmkf-fm7wg" podStartSLOduration=2.152798673 podStartE2EDuration="5.710250115s" totalImagesPullingTime="3.557451442s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:42:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:42:32.709542985 +0000 UTC m=+39.823485299" watchObservedRunningTime="2026-10-30 21:42:32.710250115 +0000 UTC m=+39.824192431"
Oct 30 21:42:33.597968 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:33.597914    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-9st74" event={"ID":"10d2e5ae-c1c4-400c-8c67-3aebb6caaa6a","Type":"ContainerStarted","Data":"fea86070a3fdd57a77c2e5e144ef9555926cf92e5decbfa1b2fb7bbf53e0fdbd"}
Oct 30 21:42:33.711621 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:33.711567    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-9st74" podStartSLOduration=35.017378833 podStartE2EDuration="39.711553943s" totalImagesPullingTime="4.69417511s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:41:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:42:33.710651978 +0000 UTC m=+40.824594291" watchObservedRunningTime="2026-10-30 21:42:33.711553943 +0000 UTC m=+40.825496255"
Oct 30 21:42:34.166649 ip-10-0-140-18 kubenswrapper[2580]: I1030 21:42:34.166553    2580 dynamic_cafile_content.go:175] "Shutting down controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 30 21:42:34.166697 ip-10-0-140-18 systemd[1]: Stopping Kubernetes Kubelet...
Oct 30 21:42:34.185390 ip-10-0-140-18 systemd[1]: kubelet.service: Deactivated successfully.
Oct 30 21:42:34.185563 ip-10-0-140-18 systemd[1]: Stopped Kubernetes Kubelet.
Oct 30 21:42:34.185611 ip-10-0-140-18 systemd[1]: kubelet.service: Consumed 2.527s CPU time.
Oct 30 21:42:34.195544 ip-10-0-140-18 systemd[1]: Starting Kubernetes Kubelet...
Oct 30 21:42:34.275400 ip-10-0-140-18 kubenswrapper[4473]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 21:42:34.275400 ip-10-0-140-18 kubenswrapper[4473]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Oct 30 21:42:34.275400 ip-10-0-140-18 kubenswrapper[4473]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 21:42:34.275400 ip-10-0-140-18 kubenswrapper[4473]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 21:42:34.277482 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.277397    4473 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:42:34.277482 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.277483    4473 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:42:34.277557 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.277487    4473 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:42:34.277557 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.277490    4473 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:42:34.277557 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.277493    4473 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:42:34.277557 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.277496    4473 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:42:34.277557 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.277498    4473 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:42:34.277557 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.277501    4473 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:42:34.277557 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.277504    4473 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:42:34.277557 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.277506    4473 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:42:34.277557 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.277509    4473 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:42:34.277557 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.277515    4473 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:42:34.277557 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.277518    4473 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:42:34.277557 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.277520    4473 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:42:34.277557 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.277525    4473 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:42:34.277557 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.277530    4473 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:42:34.277557 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.277533    4473 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:42:34.277557 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.277538    4473 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:42:34.277557 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.277541    4473 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:42:34.277557 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.277545    4473 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:42:34.277557 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.277547    4473 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:42:34.277557 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.277550    4473 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:42:34.278092 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.277552    4473 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:42:34.278092 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.277555    4473 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:42:34.278092 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.277558    4473 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:42:34.278092 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.277560    4473 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:42:34.278092 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.277563    4473 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:42:34.278092 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.277566    4473 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:42:34.278092 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.277569    4473 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:42:34.278092 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.277572    4473 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:42:34.278092 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.277575    4473 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:42:34.278092 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.277577    4473 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:42:34.278092 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.277580    4473 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:42:34.278092 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.277582    4473 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:42:34.278092 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.277586    4473 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:42:34.278092 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.277588    4473 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:42:34.278092 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.277591    4473 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:42:34.278092 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.277593    4473 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:42:34.278092 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.277595    4473 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:42:34.278092 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.277598    4473 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:42:34.278092 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.277600    4473 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:42:34.278092 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.277603    4473 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:42:34.278662 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.277605    4473 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:42:34.278662 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.277609    4473 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:42:34.278662 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.277613    4473 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:42:34.278662 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.277617    4473 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:42:34.278662 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.277620    4473 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:42:34.278662 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.277622    4473 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:42:34.278662 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.277624    4473 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:42:34.278662 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.277627    4473 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:42:34.278662 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.277630    4473 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:42:34.278662 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.277633    4473 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:42:34.278662 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.277635    4473 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:42:34.278662 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.277638    4473 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:42:34.278662 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.277640    4473 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:42:34.278662 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.277643    4473 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:42:34.278662 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.277646    4473 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:42:34.278662 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.277648    4473 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:42:34.278662 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.277651    4473 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:42:34.278662 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.277654    4473 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:42:34.278662 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.277656    4473 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:42:34.278662 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.277659    4473 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:42:34.279475 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.277662    4473 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:42:34.279475 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.277664    4473 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:42:34.279475 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.277667    4473 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:42:34.279475 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.277669    4473 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:42:34.279475 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.277672    4473 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:42:34.279475 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.277675    4473 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:42:34.279475 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.277677    4473 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:42:34.279475 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.277680    4473 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:42:34.279475 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.277682    4473 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:42:34.279475 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.277684    4473 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:42:34.279475 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.277687    4473 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:42:34.279475 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.277690    4473 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:42:34.279475 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.277692    4473 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:42:34.279475 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.277694    4473 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:42:34.279475 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.277697    4473 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:42:34.279475 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.277700    4473 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:42:34.279475 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.277703    4473 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:42:34.279475 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.277705    4473 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:42:34.279475 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.277708    4473 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:42:34.279983 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.277711    4473 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:42:34.279983 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.277714    4473 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:42:34.279983 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.277716    4473 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:42:34.279983 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.277719    4473 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:42:34.279983 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.277721    4473 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:42:34.279983 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.277724    4473 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:42:34.279983 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.277726    4473 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:42:34.279983 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.277729    4473 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:42:34.279983 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.277731    4473 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:42:34.279983 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.277734    4473 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:42:34.279983 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.277736    4473 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:42:34.279983 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.277739    4473 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:42:34.279983 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.277742    4473 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:42:34.279983 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.277744    4473 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:42:34.279983 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.277747    4473 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:42:34.279983 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.277749    4473 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:42:34.279983 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.277752    4473 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:42:34.279983 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.277755    4473 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:42:34.279983 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.277758    4473 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:42:34.280495 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.277761    4473 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:42:34.280495 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.277763    4473 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:42:34.280495 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.277766    4473 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:42:34.280495 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.277768    4473 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:42:34.280495 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.277770    4473 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:42:34.280495 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.277773    4473 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:42:34.280495 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.279362    4473 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:42:34.280495 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.279386    4473 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:42:34.280495 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.279397    4473 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:42:34.280495 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.279401    4473 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:42:34.280495 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.279406    4473 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:42:34.280495 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.279411    4473 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:42:34.280495 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.279417    4473 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:42:34.280495 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.279425    4473 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:42:34.280495 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.279429    4473 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:42:34.280495 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.279434    4473 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:42:34.280495 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.279438    4473 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:42:34.280495 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.279443    4473 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:42:34.280495 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.279447    4473 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:42:34.280998 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.279451    4473 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:42:34.280998 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.279460    4473 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:42:34.280998 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.279465    4473 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:42:34.280998 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.279482    4473 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:42:34.280998 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.279488    4473 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:42:34.280998 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.279528    4473 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:42:34.280998 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.279670    4473 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:42:34.280998 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.279675    4473 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:42:34.280998 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.279680    4473 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:42:34.280998 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.279683    4473 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:42:34.280998 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.279687    4473 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:42:34.280998 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.279689    4473 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:42:34.280998 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.279692    4473 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:42:34.280998 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.279695    4473 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:42:34.280998 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.279698    4473 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:42:34.280998 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.279701    4473 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:42:34.280998 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.279703    4473 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:42:34.280998 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.279706    4473 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:42:34.280998 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.279709    4473 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:42:34.280998 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.279712    4473 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:42:34.281509 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.279714    4473 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:42:34.281509 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.279718    4473 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:42:34.281509 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.279721    4473 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:42:34.281509 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.279724    4473 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:42:34.281509 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.279727    4473 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:42:34.281509 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.279730    4473 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:42:34.281509 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.279733    4473 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:42:34.281509 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.279735    4473 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:42:34.281509 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.279738    4473 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:42:34.281509 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.279741    4473 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:42:34.281509 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.279744    4473 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:42:34.281509 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.279746    4473 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:42:34.281509 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.279749    4473 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:42:34.281509 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.279751    4473 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:42:34.281509 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.279754    4473 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:42:34.281509 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.279757    4473 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:42:34.281509 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.279760    4473 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:42:34.281509 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.279763    4473 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:42:34.281509 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.279766    4473 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:42:34.281509 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.279768    4473 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:42:34.282038 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.279771    4473 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:42:34.282038 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.279774    4473 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:42:34.282038 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.279779    4473 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:42:34.282038 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.279782    4473 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:42:34.282038 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.279784    4473 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:42:34.282038 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.279787    4473 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:42:34.282038 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.279790    4473 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:42:34.282038 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.279793    4473 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:42:34.282038 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.279795    4473 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:42:34.282038 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.279798    4473 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:42:34.282038 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.279801    4473 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:42:34.282038 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.279804    4473 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:42:34.282038 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.279807    4473 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:42:34.282038 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.279810    4473 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:42:34.282038 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.279812    4473 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:42:34.282038 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.279815    4473 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:42:34.282038 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.279817    4473 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:42:34.282038 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.279821    4473 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:42:34.282038 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.279823    4473 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:42:34.282505 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.279826    4473 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:42:34.282505 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.279829    4473 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:42:34.282505 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.279831    4473 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:42:34.282505 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.279834    4473 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:42:34.282505 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.279837    4473 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:42:34.282505 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.279840    4473 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:42:34.282505 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.279843    4473 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:42:34.282505 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.279846    4473 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:42:34.282505 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.279848    4473 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:42:34.282505 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.279851    4473 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:42:34.282505 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.279853    4473 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:42:34.282505 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.279856    4473 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:42:34.282505 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.279859    4473 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:42:34.282505 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.279862    4473 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:42:34.282505 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.279864    4473 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:42:34.282505 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.279867    4473 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:42:34.282505 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.279869    4473 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:42:34.282505 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.279872    4473 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:42:34.282505 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.279875    4473 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:42:34.282993 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.279877    4473 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:42:34.282993 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.279880    4473 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:42:34.282993 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.279882    4473 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:42:34.282993 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.279884    4473 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:42:34.282993 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.279887    4473 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:42:34.282993 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.279889    4473 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:42:34.282993 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.279892    4473 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:42:34.282993 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.279894    4473 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:42:34.282993 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.279899    4473 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:42:34.282993 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.279902    4473 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:42:34.282993 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.279905    4473 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:42:34.282993 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.279909    4473 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:42:34.282993 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.279911    4473 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:42:34.282993 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.279914    4473 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:42:34.282993 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.279937    4473 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:42:34.282993 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.280459    4473 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:42:34.282993 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.280464    4473 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:42:34.282993 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.280467    4473 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:42:34.282993 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.280472    4473 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:42:34.283448 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.280476    4473 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:42:34.283448 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.280479    4473 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:42:34.283448 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.280482    4473 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:42:34.283448 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.280485    4473 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:42:34.283448 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.280488    4473 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:42:34.283448 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.280491    4473 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:42:34.283448 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.280494    4473 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:42:34.283448 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.280497    4473 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:42:34.283448 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.280500    4473 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:42:34.283448 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.280502    4473 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:42:34.283448 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.280505    4473 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:42:34.283448 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.280508    4473 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:42:34.283448 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.280511    4473 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:42:34.283448 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.280513    4473 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:42:34.283448 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.280516    4473 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:42:34.283448 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.280518    4473 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:42:34.283448 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.280520    4473 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:42:34.283448 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.280523    4473 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:42:34.283448 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.280525    4473 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:42:34.283448 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.280528    4473 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:42:34.284059 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.280531    4473 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:42:34.284059 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.280533    4473 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:42:34.284059 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.280536    4473 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:42:34.284059 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.280539    4473 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:42:34.284059 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.280541    4473 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:42:34.284059 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.280544    4473 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:42:34.284059 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.280547    4473 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:42:34.284059 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.280549    4473 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:42:34.284059 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.280552    4473 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:42:34.284059 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.280555    4473 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:42:34.284059 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.280557    4473 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:42:34.284059 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.280560    4473 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:42:34.284059 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.280562    4473 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:42:34.284059 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.280565    4473 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:42:34.284059 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.280567    4473 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:42:34.284059 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.280570    4473 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:42:34.284059 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.280572    4473 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:42:34.284059 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.280574    4473 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:42:34.284059 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.280578    4473 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:42:34.284607 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.280580    4473 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:42:34.284607 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.280599    4473 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:42:34.284607 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.280604    4473 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:42:34.284607 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.280608    4473 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:42:34.284607 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.280612    4473 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:42:34.284607 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.280616    4473 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:42:34.284607 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.280619    4473 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:42:34.284607 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.280622    4473 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:42:34.284607 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.280625    4473 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:42:34.284607 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.280627    4473 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:42:34.284607 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.280630    4473 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:42:34.284607 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.280632    4473 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:42:34.284607 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.280635    4473 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:42:34.284607 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.280638    4473 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:42:34.284607 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.280641    4473 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:42:34.284607 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.280643    4473 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:42:34.284607 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.280646    4473 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:42:34.284607 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.280649    4473 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:42:34.284607 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.280651    4473 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:42:34.284607 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.280654    4473 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:42:34.285150 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.280657    4473 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:42:34.285150 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.280659    4473 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:42:34.285150 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.280662    4473 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:42:34.285150 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.280664    4473 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:42:34.285150 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.280667    4473 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:42:34.285150 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.280669    4473 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:42:34.285150 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.280672    4473 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:42:34.285150 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.280674    4473 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:42:34.285150 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.280677    4473 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:42:34.285150 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.280679    4473 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:42:34.285150 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.280682    4473 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:42:34.285150 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.280684    4473 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:42:34.285150 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.280687    4473 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:42:34.285150 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.280690    4473 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:42:34.285150 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.280693    4473 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:42:34.285150 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.280696    4473 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:42:34.285150 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.280698    4473 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:42:34.285150 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.280701    4473 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:42:34.285150 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.280703    4473 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:42:34.285150 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.280705    4473 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:42:34.285641 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.280708    4473 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:42:34.285641 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.280710    4473 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:42:34.285641 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.280713    4473 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:42:34.285641 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.280716    4473 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:42:34.285641 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.280718    4473 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:42:34.285641 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.280722    4473 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:42:34.285641 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.280724    4473 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:42:34.285641 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.280727    4473 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:42:34.285641 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.280730    4473 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:42:34.285641 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.280732    4473 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:42:34.285641 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.280734    4473 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:42:34.285641 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.280738    4473 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:42:34.285641 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.280740    4473 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:42:34.285641 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.280743    4473 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:42:34.285641 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.280746    4473 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:42:34.285641 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.280748    4473 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:42:34.285641 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.280751    4473 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:42:34.285641 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.280755    4473 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:42:34.285641 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.280757    4473 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:42:34.285641 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.280760    4473 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:42:34.286200 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.280762    4473 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:42:34.286200 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.280765    4473 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:42:34.286200 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.280767    4473 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:42:34.286200 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.280859    4473 flags.go:64] FLAG: --address="0.0.0.0"
Oct 30 21:42:34.286200 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.280866    4473 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Oct 30 21:42:34.286200 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.280873    4473 flags.go:64] FLAG: --anonymous-auth="true"
Oct 30 21:42:34.286200 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.280878    4473 flags.go:64] FLAG: --application-metrics-count-limit="100"
Oct 30 21:42:34.286200 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.280883    4473 flags.go:64] FLAG: --authentication-token-webhook="false"
Oct 30 21:42:34.286200 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.280886    4473 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Oct 30 21:42:34.286200 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.280891    4473 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Oct 30 21:42:34.286200 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.280895    4473 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Oct 30 21:42:34.286200 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.280898    4473 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Oct 30 21:42:34.286200 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.280901    4473 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Oct 30 21:42:34.286200 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.280904    4473 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Oct 30 21:42:34.286200 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.280908    4473 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Oct 30 21:42:34.286200 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.280911    4473 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Oct 30 21:42:34.286200 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.280914    4473 flags.go:64] FLAG: --cgroup-root=""
Oct 30 21:42:34.286200 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.280939    4473 flags.go:64] FLAG: --cgroups-per-qos="true"
Oct 30 21:42:34.286200 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.280945    4473 flags.go:64] FLAG: --client-ca-file=""
Oct 30 21:42:34.286200 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.280949    4473 flags.go:64] FLAG: --cloud-provider="external"
Oct 30 21:42:34.286200 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.280952    4473 flags.go:64] FLAG: --cluster-dns="[]"
Oct 30 21:42:34.286200 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.280955    4473 flags.go:64] FLAG: --cluster-domain=""
Oct 30 21:42:34.286200 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.280958    4473 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Oct 30 21:42:34.286861 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.280962    4473 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Oct 30 21:42:34.286861 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.280965    4473 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Oct 30 21:42:34.286861 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.280968    4473 flags.go:64] FLAG: --container-log-max-files="5"
Oct 30 21:42:34.286861 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.280972    4473 flags.go:64] FLAG: --container-log-max-size="10Mi"
Oct 30 21:42:34.286861 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.280975    4473 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Oct 30 21:42:34.286861 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.280978    4473 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Oct 30 21:42:34.286861 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.280982    4473 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Oct 30 21:42:34.286861 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.280985    4473 flags.go:64] FLAG: --contention-profiling="false"
Oct 30 21:42:34.286861 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.280987    4473 flags.go:64] FLAG: --cpu-cfs-quota="true"
Oct 30 21:42:34.286861 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.280990    4473 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Oct 30 21:42:34.286861 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.280993    4473 flags.go:64] FLAG: --cpu-manager-policy="none"
Oct 30 21:42:34.286861 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.280996    4473 flags.go:64] FLAG: --cpu-manager-policy-options=""
Oct 30 21:42:34.286861 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.281001    4473 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Oct 30 21:42:34.286861 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.281004    4473 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Oct 30 21:42:34.286861 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.281007    4473 flags.go:64] FLAG: --enable-debugging-handlers="true"
Oct 30 21:42:34.286861 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.281017    4473 flags.go:64] FLAG: --enable-load-reader="false"
Oct 30 21:42:34.286861 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.281022    4473 flags.go:64] FLAG: --enable-server="true"
Oct 30 21:42:34.286861 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.281026    4473 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Oct 30 21:42:34.286861 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.281039    4473 flags.go:64] FLAG: --event-burst="100"
Oct 30 21:42:34.286861 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.281043    4473 flags.go:64] FLAG: --event-qps="50"
Oct 30 21:42:34.286861 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.281045    4473 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Oct 30 21:42:34.286861 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.281048    4473 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Oct 30 21:42:34.286861 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.281051    4473 flags.go:64] FLAG: --eviction-hard=""
Oct 30 21:42:34.286861 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.281056    4473 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Oct 30 21:42:34.287522 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.281059    4473 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Oct 30 21:42:34.287522 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.281063    4473 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Oct 30 21:42:34.287522 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.281067    4473 flags.go:64] FLAG: --eviction-soft=""
Oct 30 21:42:34.287522 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.281070    4473 flags.go:64] FLAG: --eviction-soft-grace-period=""
Oct 30 21:42:34.287522 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.281073    4473 flags.go:64] FLAG: --exit-on-lock-contention="false"
Oct 30 21:42:34.287522 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.281076    4473 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Oct 30 21:42:34.287522 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.281079    4473 flags.go:64] FLAG: --experimental-mounter-path=""
Oct 30 21:42:34.287522 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.281082    4473 flags.go:64] FLAG: --fail-cgroupv1="true"
Oct 30 21:42:34.287522 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.281085    4473 flags.go:64] FLAG: --fail-swap-on="true"
Oct 30 21:42:34.287522 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.281087    4473 flags.go:64] FLAG: --feature-gates=""
Oct 30 21:42:34.287522 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.281098    4473 flags.go:64] FLAG: --file-check-frequency="20s"
Oct 30 21:42:34.287522 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.281101    4473 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Oct 30 21:42:34.287522 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.281105    4473 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Oct 30 21:42:34.287522 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.281108    4473 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Oct 30 21:42:34.287522 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.281111    4473 flags.go:64] FLAG: --healthz-port="10248"
Oct 30 21:42:34.287522 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.281114    4473 flags.go:64] FLAG: --help="false"
Oct 30 21:42:34.287522 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.281117    4473 flags.go:64] FLAG: --hostname-override="ip-10-0-140-18.us-west-1.compute.internal"
Oct 30 21:42:34.287522 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.281120    4473 flags.go:64] FLAG: --housekeeping-interval="10s"
Oct 30 21:42:34.287522 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.281123    4473 flags.go:64] FLAG: --http-check-frequency="20s"
Oct 30 21:42:34.287522 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.281126    4473 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Oct 30 21:42:34.287522 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.281129    4473 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Oct 30 21:42:34.287522 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.281133    4473 flags.go:64] FLAG: --image-gc-high-threshold="85"
Oct 30 21:42:34.287522 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.281136    4473 flags.go:64] FLAG: --image-gc-low-threshold="80"
Oct 30 21:42:34.288113 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.281138    4473 flags.go:64] FLAG: --image-service-endpoint=""
Oct 30 21:42:34.288113 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.281142    4473 flags.go:64] FLAG: --kernel-memcg-notification="false"
Oct 30 21:42:34.288113 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.281145    4473 flags.go:64] FLAG: --kube-api-burst="100"
Oct 30 21:42:34.288113 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.281148    4473 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Oct 30 21:42:34.288113 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.281151    4473 flags.go:64] FLAG: --kube-api-qps="50"
Oct 30 21:42:34.288113 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.281154    4473 flags.go:64] FLAG: --kube-reserved=""
Oct 30 21:42:34.288113 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.281157    4473 flags.go:64] FLAG: --kube-reserved-cgroup=""
Oct 30 21:42:34.288113 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.281160    4473 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Oct 30 21:42:34.288113 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.281166    4473 flags.go:64] FLAG: --kubelet-cgroups=""
Oct 30 21:42:34.288113 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.281169    4473 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Oct 30 21:42:34.288113 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.281171    4473 flags.go:64] FLAG: --lock-file=""
Oct 30 21:42:34.288113 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.281174    4473 flags.go:64] FLAG: --log-cadvisor-usage="false"
Oct 30 21:42:34.288113 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.281177    4473 flags.go:64] FLAG: --log-flush-frequency="5s"
Oct 30 21:42:34.288113 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.281180    4473 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Oct 30 21:42:34.288113 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.281186    4473 flags.go:64] FLAG: --log-json-split-stream="false"
Oct 30 21:42:34.288113 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.281189    4473 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Oct 30 21:42:34.288113 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.281192    4473 flags.go:64] FLAG: --log-text-split-stream="false"
Oct 30 21:42:34.288113 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.281195    4473 flags.go:64] FLAG: --logging-format="text"
Oct 30 21:42:34.288113 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.281197    4473 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Oct 30 21:42:34.288113 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.281201    4473 flags.go:64] FLAG: --make-iptables-util-chains="true"
Oct 30 21:42:34.288113 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.281204    4473 flags.go:64] FLAG: --manifest-url=""
Oct 30 21:42:34.288113 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.281207    4473 flags.go:64] FLAG: --manifest-url-header=""
Oct 30 21:42:34.288113 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.281212    4473 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Oct 30 21:42:34.288113 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.281215    4473 flags.go:64] FLAG: --max-open-files="1000000"
Oct 30 21:42:34.288113 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.281219    4473 flags.go:64] FLAG: --max-pods="110"
Oct 30 21:42:34.288750 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.281222    4473 flags.go:64] FLAG: --maximum-dead-containers="-1"
Oct 30 21:42:34.288750 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.281225    4473 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Oct 30 21:42:34.288750 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.281228    4473 flags.go:64] FLAG: --memory-manager-policy="None"
Oct 30 21:42:34.288750 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.281231    4473 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Oct 30 21:42:34.288750 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.281234    4473 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Oct 30 21:42:34.288750 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.281237    4473 flags.go:64] FLAG: --node-ip="0.0.0.0"
Oct 30 21:42:34.288750 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.281240    4473 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Oct 30 21:42:34.288750 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.281247    4473 flags.go:64] FLAG: --node-status-max-images="50"
Oct 30 21:42:34.288750 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.281251    4473 flags.go:64] FLAG: --node-status-update-frequency="10s"
Oct 30 21:42:34.288750 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.281254    4473 flags.go:64] FLAG: --oom-score-adj="-999"
Oct 30 21:42:34.288750 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.281257    4473 flags.go:64] FLAG: --pod-cidr=""
Oct 30 21:42:34.288750 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.281260    4473 flags.go:64] FLAG: --pod-manifest-path=""
Oct 30 21:42:34.288750 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.281263    4473 flags.go:64] FLAG: --pod-max-pids="-1"
Oct 30 21:42:34.288750 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.281267    4473 flags.go:64] FLAG: --pods-per-core="0"
Oct 30 21:42:34.288750 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.281270    4473 flags.go:64] FLAG: --port="10250"
Oct 30 21:42:34.288750 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.281275    4473 flags.go:64] FLAG: --protect-kernel-defaults="false"
Oct 30 21:42:34.288750 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.281278    4473 flags.go:64] FLAG: --provider-id="aws:///us-west-1a/i-0667f248f24227d44"
Oct 30 21:42:34.288750 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.281281    4473 flags.go:64] FLAG: --qos-reserved=""
Oct 30 21:42:34.288750 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.281284    4473 flags.go:64] FLAG: --read-only-port="10255"
Oct 30 21:42:34.288750 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.281287    4473 flags.go:64] FLAG: --register-node="true"
Oct 30 21:42:34.288750 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.281290    4473 flags.go:64] FLAG: --register-with-taints=""
Oct 30 21:42:34.288750 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.281294    4473 flags.go:64] FLAG: --registry-burst="10"
Oct 30 21:42:34.288750 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.281297    4473 flags.go:64] FLAG: --registry-qps="5"
Oct 30 21:42:34.288750 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.281300    4473 flags.go:64] FLAG: --reserved-cpus=""
Oct 30 21:42:34.288750 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.281303    4473 flags.go:64] FLAG: --reserved-memory=""
Oct 30 21:42:34.288750 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.281306    4473 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Oct 30 21:42:34.289457 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.281309    4473 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Oct 30 21:42:34.289457 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.281313    4473 flags.go:64] FLAG: --rotate-certificates="false"
Oct 30 21:42:34.289457 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.281316    4473 flags.go:64] FLAG: --rotate-server-certificates="false"
Oct 30 21:42:34.289457 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.281318    4473 flags.go:64] FLAG: --runonce="false"
Oct 30 21:42:34.289457 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.281321    4473 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Oct 30 21:42:34.289457 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.281324    4473 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Oct 30 21:42:34.289457 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.281327    4473 flags.go:64] FLAG: --seccomp-default="false"
Oct 30 21:42:34.289457 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.281330    4473 flags.go:64] FLAG: --serialize-image-pulls="true"
Oct 30 21:42:34.289457 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.281333    4473 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Oct 30 21:42:34.289457 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.281336    4473 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Oct 30 21:42:34.289457 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.281338    4473 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Oct 30 21:42:34.289457 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.281341    4473 flags.go:64] FLAG: --storage-driver-password="root"
Oct 30 21:42:34.289457 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.281344    4473 flags.go:64] FLAG: --storage-driver-secure="false"
Oct 30 21:42:34.289457 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.281349    4473 flags.go:64] FLAG: --storage-driver-table="stats"
Oct 30 21:42:34.289457 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.281352    4473 flags.go:64] FLAG: --storage-driver-user="root"
Oct 30 21:42:34.289457 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.281355    4473 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Oct 30 21:42:34.289457 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.281364    4473 flags.go:64] FLAG: --sync-frequency="1m0s"
Oct 30 21:42:34.289457 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.281367    4473 flags.go:64] FLAG: --system-cgroups=""
Oct 30 21:42:34.289457 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.281370    4473 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Oct 30 21:42:34.289457 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.281375    4473 flags.go:64] FLAG: --system-reserved-cgroup=""
Oct 30 21:42:34.289457 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.281377    4473 flags.go:64] FLAG: --tls-cert-file=""
Oct 30 21:42:34.289457 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.281382    4473 flags.go:64] FLAG: --tls-cipher-suites="[]"
Oct 30 21:42:34.289457 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.281388    4473 flags.go:64] FLAG: --tls-min-version=""
Oct 30 21:42:34.289457 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.281391    4473 flags.go:64] FLAG: --tls-private-key-file=""
Oct 30 21:42:34.289457 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.281393    4473 flags.go:64] FLAG: --topology-manager-policy="none"
Oct 30 21:42:34.290092 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.281396    4473 flags.go:64] FLAG: --topology-manager-policy-options=""
Oct 30 21:42:34.290092 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.281399    4473 flags.go:64] FLAG: --topology-manager-scope="container"
Oct 30 21:42:34.290092 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.281402    4473 flags.go:64] FLAG: --v="2"
Oct 30 21:42:34.290092 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.281406    4473 flags.go:64] FLAG: --version="false"
Oct 30 21:42:34.290092 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.281410    4473 flags.go:64] FLAG: --vmodule=""
Oct 30 21:42:34.290092 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.281414    4473 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Oct 30 21:42:34.290092 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.281418    4473 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Oct 30 21:42:34.290092 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.281575    4473 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:42:34.290092 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.281579    4473 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:42:34.290092 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.281582    4473 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:42:34.290092 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.281587    4473 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:42:34.290092 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.281590    4473 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:42:34.290092 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.281593    4473 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:42:34.290092 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.281596    4473 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:42:34.290092 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.281598    4473 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:42:34.290092 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.281601    4473 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:42:34.290092 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.281604    4473 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:42:34.290092 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.281606    4473 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:42:34.290092 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.281609    4473 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:42:34.290092 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.281612    4473 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:42:34.290092 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.281616    4473 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:42:34.290594 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.281618    4473 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:42:34.290594 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.281621    4473 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:42:34.290594 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.281624    4473 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:42:34.290594 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.281627    4473 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:42:34.290594 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.281629    4473 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:42:34.290594 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.281632    4473 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:42:34.290594 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.281634    4473 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:42:34.290594 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.281638    4473 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:42:34.290594 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.281640    4473 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:42:34.290594 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.281643    4473 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:42:34.290594 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.281645    4473 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:42:34.290594 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.281648    4473 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:42:34.290594 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.281650    4473 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:42:34.290594 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.281653    4473 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:42:34.290594 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.281655    4473 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:42:34.290594 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.281658    4473 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:42:34.290594 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.281660    4473 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:42:34.290594 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.281662    4473 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:42:34.290594 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.281665    4473 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:42:34.290594 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.281667    4473 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:42:34.291102 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.281670    4473 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:42:34.291102 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.281672    4473 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:42:34.291102 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.281674    4473 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:42:34.291102 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.281677    4473 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:42:34.291102 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.281679    4473 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:42:34.291102 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.281682    4473 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:42:34.291102 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.281684    4473 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:42:34.291102 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.281687    4473 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:42:34.291102 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.281689    4473 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:42:34.291102 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.281692    4473 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:42:34.291102 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.281694    4473 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:42:34.291102 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.281698    4473 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:42:34.291102 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.281702    4473 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:42:34.291102 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.281705    4473 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:42:34.291102 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.281708    4473 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:42:34.291102 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.281711    4473 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:42:34.291102 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.281715    4473 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:42:34.291102 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.281718    4473 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:42:34.291102 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.281720    4473 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:42:34.291102 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.281723    4473 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:42:34.291653 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.281726    4473 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:42:34.291653 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.281728    4473 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:42:34.291653 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.281731    4473 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:42:34.291653 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.281733    4473 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:42:34.291653 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.281736    4473 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:42:34.291653 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.281738    4473 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:42:34.291653 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.281740    4473 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:42:34.291653 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.281743    4473 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:42:34.291653 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.281745    4473 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:42:34.291653 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.281748    4473 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:42:34.291653 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.281750    4473 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:42:34.291653 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.281752    4473 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:42:34.291653 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.281755    4473 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:42:34.291653 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.281757    4473 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:42:34.291653 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.281760    4473 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:42:34.291653 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.281762    4473 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:42:34.291653 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.281764    4473 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:42:34.291653 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.281767    4473 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:42:34.291653 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.281769    4473 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:42:34.291653 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.281772    4473 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:42:34.292296 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.281774    4473 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:42:34.292296 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.281777    4473 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:42:34.292296 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.281779    4473 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:42:34.292296 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.281785    4473 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:42:34.292296 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.281789    4473 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:42:34.292296 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.281793    4473 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:42:34.292296 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.281796    4473 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:42:34.292296 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.281799    4473 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:42:34.292296 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.281802    4473 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:42:34.292296 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.281809    4473 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:42:34.292296 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.281820    4473 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:42:34.292296 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.281824    4473 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:42:34.292296 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.281827    4473 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:42:34.292296 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.281829    4473 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:42:34.292296 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.281832    4473 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:42:34.292296 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.281835    4473 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:42:34.292296 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.281837    4473 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:42:34.292296 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.281840    4473 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:42:34.292296 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.281842    4473 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:42:34.292957 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.281845    4473 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:42:34.292957 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.281847    4473 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:42:34.292957 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.281850    4473 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:42:34.292957 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.281852    4473 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:42:34.292957 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.281855    4473 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:42:34.292957 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.281857    4473 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:42:34.292957 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.281860    4473 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:42:34.292957 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.281862    4473 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:42:34.292957 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.281865    4473 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:42:34.292957 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.281868    4473 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:42:34.292957 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.281870    4473 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:42:34.292957 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.281872    4473 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:42:34.292957 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.281875    4473 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:42:34.292957 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.281903    4473 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 21:42:34.292957 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.281960    4473 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:42:34.292957 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.281965    4473 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:42:34.293361 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.281970    4473 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:42:34.293361 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.281973    4473 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:42:34.293361 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.281976    4473 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:42:34.293361 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.281978    4473 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:42:34.293361 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.281981    4473 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:42:34.293361 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.281983    4473 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:42:34.293361 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.281986    4473 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:42:34.293361 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.281990    4473 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:42:34.293361 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.281993    4473 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:42:34.293361 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.281995    4473 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:42:34.293361 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.282000    4473 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:42:34.293361 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.282002    4473 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:42:34.293361 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.282005    4473 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:42:34.293361 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.282007    4473 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:42:34.293361 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.282010    4473 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:42:34.293361 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.282013    4473 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:42:34.293361 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.282017    4473 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:42:34.293361 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.282020    4473 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:42:34.293361 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.282023    4473 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:42:34.294058 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.282025    4473 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:42:34.294058 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.282028    4473 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:42:34.294058 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.282030    4473 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:42:34.294058 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.282033    4473 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:42:34.294058 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.282036    4473 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:42:34.294058 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.282038    4473 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:42:34.294058 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.282041    4473 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:42:34.294058 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.282044    4473 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:42:34.294058 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.282047    4473 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:42:34.294058 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.282049    4473 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:42:34.294058 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.282052    4473 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:42:34.294058 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.282054    4473 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:42:34.294058 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.282057    4473 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:42:34.294058 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.282061    4473 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:42:34.294058 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.282063    4473 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:42:34.294058 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.282066    4473 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:42:34.294058 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.282068    4473 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:42:34.294058 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.282071    4473 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:42:34.294058 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.282073    4473 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:42:34.294058 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.282076    4473 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:42:34.294873 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.282078    4473 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:42:34.294873 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.282082    4473 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:42:34.294873 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.282084    4473 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:42:34.294873 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.282089    4473 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:42:34.294873 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.282091    4473 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:42:34.294873 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.282094    4473 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:42:34.294873 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.282096    4473 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:42:34.294873 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.282100    4473 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:42:34.294873 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.282104    4473 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:42:34.294873 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.282107    4473 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:42:34.294873 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.282110    4473 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:42:34.294873 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.282112    4473 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:42:34.294873 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.282115    4473 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:42:34.294873 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.282117    4473 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:42:34.294873 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.282120    4473 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:42:34.294873 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.282123    4473 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:42:34.294873 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.282126    4473 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:42:34.294873 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.282128    4473 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:42:34.294873 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.282131    4473 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:42:34.295672 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.282133    4473 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:42:34.295672 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.282136    4473 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:42:34.295672 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.282138    4473 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:42:34.295672 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.282141    4473 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:42:34.295672 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.282143    4473 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:42:34.295672 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.282145    4473 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:42:34.295672 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.282150    4473 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:42:34.295672 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.282152    4473 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:42:34.295672 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.282155    4473 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:42:34.295672 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.282157    4473 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:42:34.295672 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.282160    4473 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:42:34.295672 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.282162    4473 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:42:34.295672 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.282165    4473 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:42:34.295672 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.282167    4473 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:42:34.295672 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.282170    4473 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:42:34.295672 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.282173    4473 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:42:34.295672 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.282177    4473 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:42:34.295672 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.282180    4473 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:42:34.295672 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.282182    4473 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:42:34.295672 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.282185    4473 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:42:34.296194 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.282187    4473 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:42:34.296194 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.282190    4473 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:42:34.296194 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.282193    4473 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:42:34.296194 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.282196    4473 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:42:34.296194 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.282198    4473 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:42:34.296194 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.282201    4473 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:42:34.296194 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.282203    4473 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:42:34.296194 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.282206    4473 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:42:34.296194 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.282209    4473 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:42:34.296194 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.282212    4473 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:42:34.296194 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.282215    4473 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:42:34.296194 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.282217    4473 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:42:34.296194 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.282220    4473 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:42:34.296194 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.282223    4473 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:42:34.296194 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.282225    4473 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:42:34.296194 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.282228    4473 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:42:34.296194 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.282230    4473 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:42:34.296194 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.282233    4473 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:42:34.296194 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.282236    4473 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:42:34.296672 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.282239    4473 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:42:34.296672 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.282242    4473 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:42:34.296672 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.282244    4473 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:42:34.296672 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.282246    4473 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:42:34.296672 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.282249    4473 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:42:34.296672 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.282251    4473 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:42:34.296672 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.282254    4473 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:42:34.296672 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.286232    4473 server.go:525] "Kubelet version" kubeletVersion="v1.35.9"
Oct 30 21:42:34.296672 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.286247    4473 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Oct 30 21:42:34.296672 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.286264    4473 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Oct 30 21:42:34.296672 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.286269    4473 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Oct 30 21:42:34.296672 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286330    4473 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:42:34.296672 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286335    4473 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:42:34.296672 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286338    4473 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:42:34.296672 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286341    4473 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:42:34.296672 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286344    4473 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:42:34.296672 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286346    4473 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:42:34.296672 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286349    4473 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:42:34.296672 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286352    4473 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:42:34.296672 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286354    4473 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:42:34.297547 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286357    4473 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:42:34.297547 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286359    4473 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:42:34.297547 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286362    4473 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:42:34.297547 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286366    4473 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:42:34.297547 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286369    4473 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:42:34.297547 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286372    4473 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:42:34.297547 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286374    4473 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:42:34.297547 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286377    4473 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:42:34.297547 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286379    4473 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:42:34.297547 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286382    4473 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:42:34.297547 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286384    4473 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:42:34.297547 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286388    4473 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:42:34.297547 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286390    4473 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:42:34.297547 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286393    4473 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:42:34.297547 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286396    4473 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:42:34.297547 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286398    4473 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:42:34.297547 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286401    4473 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:42:34.297547 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286403    4473 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:42:34.297547 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286406    4473 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:42:34.298336 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286409    4473 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:42:34.298336 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286411    4473 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:42:34.298336 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286415    4473 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:42:34.298336 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286417    4473 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:42:34.298336 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286420    4473 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:42:34.298336 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286423    4473 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:42:34.298336 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286425    4473 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:42:34.298336 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286428    4473 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:42:34.298336 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286431    4473 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:42:34.298336 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286433    4473 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:42:34.298336 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286436    4473 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:42:34.298336 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286438    4473 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:42:34.298336 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286441    4473 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:42:34.298336 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286443    4473 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:42:34.298336 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286446    4473 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:42:34.298336 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286448    4473 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:42:34.298336 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286451    4473 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:42:34.298336 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286453    4473 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:42:34.298336 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286456    4473 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:42:34.298830 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286458    4473 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:42:34.298830 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286461    4473 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:42:34.298830 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286463    4473 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:42:34.298830 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286466    4473 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:42:34.298830 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286468    4473 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:42:34.298830 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286471    4473 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:42:34.298830 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286473    4473 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:42:34.298830 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286476    4473 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:42:34.298830 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286478    4473 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:42:34.298830 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286481    4473 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:42:34.298830 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286484    4473 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:42:34.298830 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286486    4473 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:42:34.298830 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286489    4473 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:42:34.298830 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286492    4473 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:42:34.298830 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286495    4473 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:42:34.298830 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286497    4473 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:42:34.298830 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286501    4473 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:42:34.298830 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286503    4473 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:42:34.298830 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286506    4473 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:42:34.298830 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286508    4473 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:42:34.299366 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286511    4473 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:42:34.299366 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286513    4473 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:42:34.299366 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286516    4473 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:42:34.299366 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286518    4473 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:42:34.299366 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286521    4473 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:42:34.299366 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286524    4473 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:42:34.299366 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286526    4473 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:42:34.299366 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286528    4473 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:42:34.299366 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286531    4473 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:42:34.299366 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286533    4473 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:42:34.299366 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286536    4473 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:42:34.299366 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286538    4473 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:42:34.299366 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286541    4473 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:42:34.299366 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286545    4473 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:42:34.299366 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286548    4473 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:42:34.299366 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286552    4473 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:42:34.299366 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286554    4473 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:42:34.299366 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286557    4473 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:42:34.299366 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286560    4473 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:42:34.299824 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286563    4473 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:42:34.299824 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286566    4473 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:42:34.299824 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286568    4473 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:42:34.299824 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286571    4473 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:42:34.299824 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286573    4473 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:42:34.299824 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286576    4473 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:42:34.299824 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286578    4473 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:42:34.299824 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286581    4473 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:42:34.299824 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286583    4473 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:42:34.299824 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286586    4473 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:42:34.299824 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286589    4473 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:42:34.299824 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286592    4473 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:42:34.299824 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286595    4473 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:42:34.299824 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286597    4473 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:42:34.299824 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286599    4473 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:42:34.299824 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286602    4473 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:42:34.299824 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286604    4473 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:42:34.299824 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286607    4473 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:42:34.299824 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286609    4473 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:42:34.299824 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286612    4473 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:42:34.300349 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.286640    4473 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 21:42:34.300349 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286776    4473 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:42:34.300349 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286780    4473 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:42:34.300349 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286783    4473 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:42:34.300349 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286785    4473 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:42:34.300349 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286788    4473 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:42:34.300349 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286790    4473 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:42:34.300349 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286793    4473 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:42:34.300349 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286796    4473 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:42:34.300349 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286798    4473 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:42:34.300349 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286801    4473 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:42:34.300349 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286803    4473 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:42:34.300349 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286806    4473 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:42:34.300349 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286809    4473 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:42:34.300349 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286813    4473 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:42:34.300349 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286816    4473 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:42:34.300737 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286819    4473 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:42:34.300737 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286821    4473 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:42:34.300737 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286824    4473 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:42:34.300737 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286827    4473 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:42:34.300737 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286829    4473 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:42:34.300737 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286832    4473 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:42:34.300737 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286835    4473 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:42:34.300737 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286838    4473 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:42:34.300737 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286840    4473 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:42:34.300737 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286843    4473 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:42:34.300737 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286845    4473 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:42:34.300737 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286849    4473 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:42:34.300737 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286853    4473 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:42:34.300737 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286857    4473 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:42:34.300737 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286861    4473 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:42:34.300737 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286864    4473 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:42:34.300737 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286867    4473 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:42:34.300737 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286870    4473 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:42:34.300737 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286873    4473 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:42:34.300737 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286875    4473 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:42:34.301293 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286878    4473 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:42:34.301293 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286881    4473 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:42:34.301293 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286883    4473 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:42:34.301293 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286886    4473 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:42:34.301293 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286888    4473 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:42:34.301293 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286891    4473 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:42:34.301293 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286893    4473 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:42:34.301293 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286895    4473 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:42:34.301293 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286898    4473 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:42:34.301293 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286900    4473 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:42:34.301293 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286905    4473 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:42:34.301293 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286907    4473 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:42:34.301293 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286910    4473 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:42:34.301293 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286912    4473 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:42:34.301293 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286937    4473 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:42:34.301293 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286941    4473 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:42:34.301293 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286944    4473 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:42:34.301293 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286946    4473 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:42:34.301293 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286950    4473 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:42:34.301293 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286952    4473 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:42:34.301776 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286956    4473 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:42:34.301776 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286958    4473 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:42:34.301776 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286961    4473 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:42:34.301776 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286963    4473 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:42:34.301776 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286966    4473 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:42:34.301776 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286968    4473 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:42:34.301776 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286971    4473 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:42:34.301776 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286973    4473 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:42:34.301776 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286976    4473 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:42:34.301776 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286978    4473 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:42:34.301776 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286980    4473 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:42:34.301776 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286983    4473 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:42:34.301776 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286985    4473 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:42:34.301776 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286988    4473 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:42:34.301776 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286990    4473 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:42:34.301776 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286993    4473 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:42:34.301776 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286995    4473 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:42:34.301776 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.286998    4473 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:42:34.301776 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.287000    4473 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:42:34.302309 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.287003    4473 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:42:34.302309 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.287006    4473 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:42:34.302309 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.287008    4473 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:42:34.302309 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.287013    4473 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:42:34.302309 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.287015    4473 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:42:34.302309 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.287018    4473 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:42:34.302309 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.287020    4473 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:42:34.302309 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.287024    4473 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:42:34.302309 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.287027    4473 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:42:34.302309 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.287030    4473 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:42:34.302309 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.287032    4473 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:42:34.302309 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.287035    4473 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:42:34.302309 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.287037    4473 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:42:34.302309 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.287040    4473 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:42:34.302309 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.287043    4473 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:42:34.302309 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.287045    4473 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:42:34.302309 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.287048    4473 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:42:34.302309 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.287050    4473 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:42:34.302309 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.287053    4473 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:42:34.302788 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.287055    4473 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:42:34.302788 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.287058    4473 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:42:34.302788 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.287061    4473 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:42:34.302788 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.287063    4473 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:42:34.302788 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.287066    4473 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:42:34.302788 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.287068    4473 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:42:34.302788 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.287070    4473 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:42:34.302788 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.287074    4473 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:42:34.302788 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.287076    4473 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:42:34.302788 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.287078    4473 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:42:34.302788 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.287081    4473 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:42:34.302788 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.287083    4473 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:42:34.302788 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.287086    4473 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:42:34.302788 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.287111    4473 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 21:42:34.302788 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.287144    4473 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:42:34.302788 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.287147    4473 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:42:34.303332 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.287151    4473 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:42:34.303332 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.287155    4473 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:42:34.303332 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.287157    4473 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:42:34.303332 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.287160    4473 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:42:34.303332 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.287163    4473 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:42:34.303332 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.287166    4473 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:42:34.303332 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.287169    4473 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:42:34.303332 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.287171    4473 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:42:34.303332 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.287175    4473 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:42:34.303332 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.287178    4473 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:42:34.303332 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.287182    4473 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:42:34.303332 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.287185    4473 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:42:34.303332 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.287189    4473 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:42:34.303332 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.287192    4473 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:42:34.303332 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.287195    4473 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:42:34.303332 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.287197    4473 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:42:34.303332 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.287200    4473 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:42:34.303332 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.287202    4473 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:42:34.303332 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.287205    4473 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:42:34.303839 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.287207    4473 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:42:34.303839 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.287210    4473 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:42:34.303839 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.287212    4473 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:42:34.303839 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.287215    4473 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:42:34.303839 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.287218    4473 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:42:34.303839 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.287220    4473 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:42:34.303839 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.287223    4473 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:42:34.303839 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.287226    4473 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:42:34.303839 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.287228    4473 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:42:34.303839 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.287231    4473 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:42:34.303839 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.287234    4473 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:42:34.303839 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.287236    4473 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:42:34.303839 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.287239    4473 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:42:34.303839 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.287241    4473 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:42:34.303839 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.287245    4473 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:42:34.303839 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.287247    4473 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:42:34.303839 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.287250    4473 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:42:34.303839 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.287253    4473 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:42:34.303839 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.287256    4473 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:42:34.303839 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.287259    4473 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:42:34.304354 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.287261    4473 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:42:34.304354 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.287264    4473 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:42:34.304354 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.287266    4473 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:42:34.304354 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.287269    4473 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:42:34.304354 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.287271    4473 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:42:34.304354 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.287274    4473 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:42:34.304354 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.287277    4473 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:42:34.304354 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.287280    4473 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:42:34.304354 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.287282    4473 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:42:34.304354 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.287285    4473 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:42:34.304354 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.287288    4473 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:42:34.304354 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.287290    4473 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:42:34.304354 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.287295    4473 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:42:34.304354 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.287299    4473 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:42:34.304354 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.287304    4473 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:42:34.304354 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.287307    4473 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:42:34.304354 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.287311    4473 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:42:34.304354 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.287314    4473 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:42:34.304354 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.287318    4473 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:42:34.304798 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.287321    4473 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:42:34.304798 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.287325    4473 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:42:34.304798 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.287328    4473 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:42:34.304798 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.287332    4473 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:42:34.304798 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.287335    4473 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:42:34.304798 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.287339    4473 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:42:34.304798 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.287343    4473 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:42:34.304798 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.287349    4473 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:42:34.304798 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.287353    4473 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:42:34.304798 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.287356    4473 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:42:34.304798 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.287361    4473 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:42:34.304798 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.287365    4473 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:42:34.304798 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.287368    4473 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:42:34.304798 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.287372    4473 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:42:34.304798 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.287376    4473 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:42:34.304798 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.287379    4473 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:42:34.304798 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.287383    4473 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:42:34.304798 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.287387    4473 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:42:34.304798 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.287391    4473 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:42:34.305384 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.287395    4473 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:42:34.305384 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.287399    4473 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:42:34.305384 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.287403    4473 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:42:34.305384 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.287407    4473 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:42:34.305384 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.287411    4473 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:42:34.305384 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.287415    4473 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:42:34.305384 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.287419    4473 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:42:34.305384 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.287424    4473 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:42:34.305384 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.287427    4473 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:42:34.305384 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.287430    4473 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:42:34.305384 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.287432    4473 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:42:34.305384 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.287435    4473 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:42:34.305384 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.287438    4473 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:42:34.305384 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.287440    4473 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:42:34.305384 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.287443    4473 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:42:34.305384 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.287446    4473 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:42:34.305384 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.287448    4473 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:42:34.305384 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.287451    4473 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:42:34.305384 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.287454    4473 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:42:34.305384 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.287457    4473 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:42:34.305941 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.287462    4473 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:42:34.305941 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.287464    4473 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:42:34.305941 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.287467    4473 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:42:34.305941 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.287471    4473 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:42:34.305941 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.287473    4473 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:42:34.305941 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.287476    4473 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:42:34.305941 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:34.287478    4473 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:42:34.305941 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.287531    4473 server.go:957] "Client rotation is on, will bootstrap in background"
Oct 30 21:42:34.305941 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.288584    4473 bootstrap.go:85] "Current kubeconfig file contents are still valid, no bootstrap necessary"
Oct 30 21:42:34.305941 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.288645    4473 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-client-current.pem"
Oct 30 21:42:34.305941 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.288866    4473 server.go:1014] "Starting client certificate rotation"
Oct 30 21:42:34.305941 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.288958    4473 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Oct 30 21:42:34.305941 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.289007    4473 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-10-29 21:36:53 +0000 UTC" deadline="2028-07-26 17:56:52.690769022 +0000 UTC"
Oct 30 21:42:34.305941 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.289034    4473 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15236h14m18.40173675s"
Oct 30 21:42:34.305941 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.289346    4473 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 30 21:42:34.305941 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.290726    4473 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 30 21:42:34.306421 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.292266    4473 log.go:25] "Validated CRI v1 runtime API"
Oct 30 21:42:34.306421 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.293772    4473 log.go:25] "Validated CRI v1 image API"
Oct 30 21:42:34.306421 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.294540    4473 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Oct 30 21:42:34.306421 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.296592    4473 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 f6aaa1c3-3c60-4fad-b668-80a24b2419cb:/dev/nvme0n1p4 f6e04ef7-2419-486f-af3a-219a87e81add:/dev/nvme0n1p3]
Oct 30 21:42:34.306596 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.296612    4473 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/20fb5e527b7a589fc2087d3651238f91f4c7354431ea9ec95b08f8d6e78d96c6/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/20fb5e527b7a589fc2087d3651238f91f4c7354431ea9ec95b08f8d6e78d96c6/userdata/shm major:0 minor:95 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/27f3fa2df241ed15c0c679361f882d1b7a1c504e1f510e4d568952e3f43917d3/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/27f3fa2df241ed15c0c679361f882d1b7a1c504e1f510e4d568952e3f43917d3/userdata/shm major:0 minor:72 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/47636ce1c9aefb50eea1cbdaf7ddbc964ffdd48cb1b0c4160ebe0fc9c9f929e9/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/47636ce1c9aefb50eea1cbdaf7ddbc964ffdd48cb1b0c4160ebe0fc9c9f929e9/userdata/shm major:0 minor:68 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/5e9d9d5b697e87a3519595619e275bc9ad494e247a8e7e1892ea4d63b50a4a56/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/5e9d9d5b697e87a3519595619e275bc9ad494e247a8e7e1892ea4d63b50a4a56/userdata/shm major:0 minor:207 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/76411d00e5efeac0f8e2cdc0c51751cb906f91e91e8b427c7cd78c85f25ebfa6/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/76411d00e5efeac0f8e2cdc0c51751cb906f91e91e8b427c7cd78c85f25ebfa6/userdata/shm major:0 minor:221 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/79db91fb4dc34e8d30ddec93a6117fc1d4443038b0f2d8fc5c72483c4e7c1e68/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/79db91fb4dc34e8d30ddec93a6117fc1d4443038b0f2d8fc5c72483c4e7c1e68/userdata/shm major:0 minor:80 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/7a14b2934bbe1c4ee21f714f16eb03e32018d69985a144c98a7450961a811688/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/7a14b2934bbe1c4ee21f714f16eb03e32018d69985a144c98a7450961a811688/userdata/shm major:0 minor:202 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/7b5fc3aef820c3083e5409913ee1bc84b01ce70c5542392f5f23ae7c3d241c67/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/7b5fc3aef820c3083e5409913ee1bc84b01ce70c5542392f5f23ae7c3d241c67/userdata/shm major:0 minor:90 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/801bef426e6476fee21829f7a500535c139dc4114bb4ef9843db0c96de15297d/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/801bef426e6476fee21829f7a500535c139dc4114bb4ef9843db0c96de15297d/userdata/shm major:0 minor:47 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/861ed8ec8169a70ec732075a3a81d7cce667a723e813db6aa2087d0be3b3bcf5/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/861ed8ec8169a70ec732075a3a81d7cce667a723e813db6aa2087d0be3b3bcf5/userdata/shm major:0 minor:86 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/c30e39faca61472a40248b9cb2c2aecc5cd89074cddd409ced6aa73a7e97ff49/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/c30e39faca61472a40248b9cb2c2aecc5cd89074cddd409ced6aa73a7e97ff49/userdata/shm major:0 minor:211 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/c829b82f78e3de9b1b8ccab3a669062ac59ed7b3e800a2ef83c3037dcc6b0b75/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/c829b82f78e3de9b1b8ccab3a669062ac59ed7b3e800a2ef83c3037dcc6b0b75/userdata/shm major:0 minor:48 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/d8fa13f6ee6c04e40723c9b7bd7388ae0cb58ff50531dfd0e89bf94ad5309b77/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/d8fa13f6ee6c04e40723c9b7bd7388ae0cb58ff50531dfd0e89bf94ad5309b77/userdata/shm major:0 minor:76 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/ea9ef8f120528befa45af4e9c5d895ac4de2f82aa8e6d58db49358916aa15775/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/ea9ef8f120528befa45af4e9c5d895ac4de2f82aa8e6d58db49358916aa15775/userdata/shm major:0 minor:217 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/ff246766b2304f22246fdd270a6b38453bc6d295d4e10b3536995b636b482eae/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/ff246766b2304f22246fdd270a6b38453bc6d295d4e10b3536995b636b482eae/userdata/shm major:0 minor:94 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0032cb99-3a37-40c6-98cb-2d40ae4a9319/volumes/kubernetes.io~secret/original-pull-secret:{mountpoint:/var/lib/kubelet/pods/0032cb99-3a37-40c6-98cb-2d40ae4a9319/volumes/kubernetes.io~secret/original-pull-secret major:0 minor:206 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/03a62eeb-c318-4ce3-b6e9-1afba98920e9/volumes/kubernetes.io~projected/kube-api-access-9bq8j:{mountpoint:/var/lib/kubelet/pods/03a62eeb-c318-4ce3-b6e9-1afba98920e9/volumes/kubernetes.io~projected/kube-api-access-9bq8j major:0 minor:63 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/06552eb8-7379-40df-b31c-a133e646afb9/volumes/kubernetes.io~secret/agent-certs:{mountpoint:/var/lib/kubelet/pods/06552eb8-7379-40df-b31c-a133e646afb9/volumes/kubernetes.io~secret/agent-certs major:0 minor:58 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0743c782-f5ad-4d3a-86e7-fc66563902c7/volumes/kubernetes.io~projected/kube-api-access-r9bdf:{mountpoint:/var/lib/kubelet/pods/0743c782-f5ad-4d3a-86e7-fc66563902c7/volumes/kubernetes.io~projected/kube-api-access-r9bdf major:0 minor:67 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/10d2e5ae-c1c4-400c-8c67-3aebb6caaa6a/volumes/kubernetes.io~projected/kube-api-access-js6zb:{mountpoint:/var/lib/kubelet/pods/10d2e5ae-c1c4-400c-8c67-3aebb6caaa6a/volumes/kubernetes.io~projected/kube-api-access-js6zb major:0 minor:209 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/2428ae10-7385-4056-aabe-bdf1b974a36b/volumes/kubernetes.io~projected/kube-api-access-5xlxd:{mountpoint:/var/lib/kubelet/pods/2428ae10-7385-4056-aabe-bdf1b974a36b/volumes/kubernetes.io~projected/kube-api-access-5xlxd major:0 minor:66 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/2634eb5b-4ce8-4742-819a-616dcd33da39/volumes/kubernetes.io~projected/kube-api-access-htnvw:{mountpoint:/var/lib/kubelet/pods/2634eb5b-4ce8-4742-819a-616dcd33da39/volumes/kubernetes.io~projected/kube-api-access-htnvw major:0 minor:62 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/32a1c660-effa-4b6c-975a-dbb68c468eda/volumes/kubernetes.io~projected/kube-api-access-zzqrt:{mountpoint:/var/lib/kubelet/pods/32a1c660-effa-4b6c-975a-dbb68c468eda/volumes/kubernetes.io~projected/kube-api-access-zzqrt major:0 minor:210 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4436ec50-e190-4c1c-8b6c-2b9b5d12cb9f/volumes/kubernetes.io~projected/kube-api-access-7vks4:{mountpoint:/var/lib/kubelet/pods/4436ec50-e190-4c1c-8b6c-2b9b5d12cb9f/volumes/kubernetes.io~projected/kube-api-access-7vks4 major:0 minor:197 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4436ec50-e190-4c1c-8b6c-2b9b5d12cb9f/volumes/kubernetes.io~secret/cert:{mountpoint:/var/lib/kubelet/pods/4436ec50-e190-4c1c-8b6c-2b9b5d12cb9f/volumes/kubernetes.io~secret/cert major:0 minor:45 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/525a0437-9a44-425b-966a-b118d91a4522/volumes/kubernetes.io~projected/kube-api-access-g98k9:{mountpoint:/var/lib/kubelet/pods/525a0437-9a44-425b-966a-b118d91a4522/volumes/kubernetes.io~projected/kube-api-access-g98k9 major:0 minor:215 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/5de0b6c7-0d1c-4732-abce-361a46df75c3/volumes/kubernetes.io~projected/kube-api-access-rfxjl:{mountpoint:/var/lib/kubelet/pods/5de0b6c7-0d1c-4732-abce-361a46df75c3/volumes/kubernetes.io~projected/kube-api-access-rfxjl major:0 minor:60 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/641ac65e-00db-4646-b03c-9e3cb0c70feb/volume-subpaths/run-systemd/ovnkube-controller/6:{mountpoint:/var/lib/kubelet/pods/641ac65e-00db-4646-b03c-9e3cb0c70feb/volume-subpaths/run-systemd/ovnkube-controller/6 major:0 minor:26 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/641ac65e-00db-4646-b03c-9e3cb0c70feb/volumes/kubernetes.io~projected/kube-api-access-9hm6j:{mountpoint:/var/lib/kubelet/pods/641ac65e-00db-4646-b03c-9e3cb0c70feb/volumes/kubernetes.io~projected/kube-api-access-9hm6j major:0 minor:64 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/641ac65e-00db-4646-b03c-9e3cb0c70feb/volumes/kubernetes.io~secret/ovn-node-metrics-cert:{mountpoint:/var/lib/kubelet/pods/641ac65e-00db-4646-b03c-9e3cb0c70feb/volumes/kubernetes.io~secret/ovn-node-metrics-cert major:0 minor:57 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/719f98aa-6247-4973-8d2c-a91aaf276984/volumes/kubernetes.io~projected/kube-api-access-pn9mk:{mountpoint:/var/lib/kubelet/pods/719f98aa-6247-4973-8d2c-a91aaf276984/volumes/kubernetes.io~projected/kube-api-access-pn9mk major:0 minor:65 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/bddede0e-ee07-4344-878d-10b967435e13/volumes/kubernetes.io~projected/kube-api-access-lnn4t:{mountpoint:/var/lib/kubelet/pods/bddede0e-ee07-4344-878d-10b967435e13/volumes/kubernetes.io~projected/kube-api-access-lnn4t major:0 minor:216 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/c67fe06a-8f45-48d0-9451-b9157f57369e/volumes/kubernetes.io~empty-dir/etc-tuned:{mountpoint:/var/lib/kubelet/pods/c67fe06a-8f45-48d0-9451-b9157f57369e/volumes/kubernetes.io~empty-dir/etc-tuned major:0 minor:56 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/c67fe06a-8f45-48d0-9451-b9157f57369e/volumes/kubernetes.io~empty-dir/tmp:{mountpoint:/var/lib/kubelet/pods/c67fe06a-8f45-48d0-9451-b9157f57369e/volumes/kubernetes.io~empty-dir/tmp major:0 minor:55 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/c67fe06a-8f45-48d0-9451-b9157f57369e/volumes/kubernetes.io~projected/kube-api-access-64s8x:{mountpoint:/var/lib/kubelet/pods/c67fe06a-8f45-48d0-9451-b9157f57369e/volumes/kubernetes.io~projected/kube-api-access-64s8x major:0 minor:61 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ccc5af13-906a-469d-9ca5-09366d1a7e93/volumes/kubernetes.io~projected/kube-api-access-b8x4j:{mountpoint:/var/lib/kubelet/pods/ccc5af13-906a-469d-9ca5-09366d1a7e93/volumes/kubernetes.io~projected/kube-api-access-b8x4j major:0 minor:59 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/f152360c-cf62-4d10-b2e5-b85ac7dfb739/volumes/kubernetes.io~projected/kube-api-access-n6qcb:{mountpoint:/var/lib/kubelet/pods/f152360c-cf62-4d10-b2e5-b85ac7dfb739/volumes/kubernetes.io~projected/kube-api-access-n6qcb major:0 minor:201 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0} overlay_0-100:{mountpoint:/var/lib/containers/storage/overlay/cc74036daf5bedf1ee500a9b2b03bda1de0dda599125af5de53aabdb67e3989e/merged major:0 minor:100 fsType:overlay blockSize:0} overlay_0-102:{mountpoint:/var/lib/containers/storage/overlay/07a4015fac37ccaa18c725c437612ca934b0b87d7d4e47697281552bf8494536/merged major:0 minor:102 fsType:overlay blockSize:0} overlay_0-104:{mountpoint:/var/lib/containers/storage/overlay/d22be5b0098503d35346d6389c6e0a3bcab5bb7d6a93c81827cfc9e8689faa67/merged major:0 minor:104 fsType:overlay blockSize:0} overlay_0-105:{mountpoint:/var/lib/containers/storage/overlay/9d4c64fdb3d2a2219d612189993684354e4adce6182611cd20a38b7c4bab4b33/merged major:0 minor:105 fsType:overlay blockSize:0} overlay_0-108:{mountpoint:/var/lib/containers/storage/overlay/fbfae57f41564021ea87b75071bfae445b6c132973d3db7252be988022c04db7/merged major:0 minor:108 fsType:overlay blockSize:0} overlay_0-119:{mountpoint:/var/lib/containers/storage/overlay/a3f80bbce821c60430afb15b9870e63d30b0f8af2bf042797dab786572454797/merged major:0 minor:119 fsType:overlay blockSize:0} overlay_0-124:{mountpoint:/var/lib/containers/storage/overlay/f0a662ab72cba31c34580725a6b074e9318d508be4e7b06ac8d3e6e72fbe937a/merged major:0 minor:124 fsType:overlay blockSize:0} overlay_0-127:{mountpoint:/var/lib/containers/storage/overlay/0ac73d8072bf2cb5c19ad9a5bef67b64522453bfb4c8098a9ada648e2d2d7ae9/merged major:0 minor:127 fsType:overlay blockSize:0} overlay_0-132:{mountpoint:/var/lib/containers/storage/overlay/3f2c68facbc50b711233b10b0502cadc80b726670d58d0925aafafaf4c014750/merged major:0 minor:132 fsType:overlay blockSize:0} overlay_0-134:{mountpoint:/var/lib/containers/storage/overlay/d52360164d2d2488d772dfcce6cee895759ab6b0976fe68b971aac173565c658/merged major:0 minor:134 fsType:overlay blockSize:0} overlay_0-139:{mountpoint:/var/lib/containers/storage/overlay/094f94fc88c1582703bd304c644b746bbeb2c5a3184d718f276a0f68c76fd061/merged major:0 minor:139 fsType:overlay blockSize:0} overlay_0-143:{mountpoint:/var/lib/containers/storage/overlay/5d2a24a7572c7efc1d25d8833e9e49686db0adf2f2b8d70aa6689400414da134/merged major:0 minor:143 fsType:overlay blockSize:0} overlay_0-152:{mountpoint:/var/lib/containers/storage/overlay/1091fb221c87263e37d017cca525d1d6fb98fa589bfb24d5e11cb0c147286e8c/merged major:0 minor:152 fsType:overlay blockSize:0} overlay_0-157:{mountpoint:/var/lib/containers/storage/overlay/624c8a36b483e3b4ab73aee78d546701b0dc73dd6a98ef91db7325384cd93506/merged major:0 minor:157 fsType:overlay blockSize:0} overlay_0-162:{mountpoint:/var/lib/containers/storage/overlay/6afb3f03491f78c3771e3db50fe96e02130691b0d20f4dc68e40af0b4485001c/merged major:0 minor:162 fsType:overlay blockSize:0} overlay_0-167:{mountpoint:/var/lib/containers/storage/overlay/984525d30c5deac037b0b17b335f45363feb030b707301a524051caa6d53977d/merged major:0 minor:167 fsType:overlay blockSize:0} overlay_0-172:{mountpoint:/var/lib/containers/storage/overlay/ad61e233867c2e76401a11ebbae815e75deca6c079f05ede81d407d717e927d0/merged major:0 minor:172 fsType:overlay blockSize:0} overlay_0-177:{mountpoint:/var/lib/containers/storage/overlay/14e60e1a34f586fc32784c54eed416290ceebe9cfbf9dbb5d4bfc1e643650a16/merged major:0 minor:177 fsType:overlay blockSize:0} overlay_0-182:{mountpoint:/var/lib/containers/storage/overlay/e898e3301413e59433c49795fed13d97e7f46299f213c4c8fcfbd23630c6d820/merged major:0 minor:182 fsType:overlay blockSize:0} overlay_0-187:{mountpoint:/var/lib/containers/storage/overlay/ab9cd31e95fc1c49f227efb10665ed90ce65b3c25efe8aa4eb5cc5cdbeb548f9/merged major:0 minor:187 fsType:overlay blockSize:0} overlay_0-192:{mountpoint:/var/lib/containers/storage/overlay/ad33cbf52c42d8b96dc9c28b5229ca714fdc54fdbd3330eb3b6b21fec7219d12/merged major:0 minor:192 fsType:overlay blockSize:0} overlay_0-204:{mountpoint:/var/lib/containers/storage/overlay/0019f04eb2efacefb97ce67478d0c12029ba3b10e91266ec1bb5ce1a8da00f1d/merged major:0 minor:204 fsType:overlay blockSize:0} overlay_0-213:{mountpoint:/var/lib/containers/storage/overlay/d6b77e64730da95f9daa5751c8e3c48b7c1af04e8117130af5ede9135c307c53/merged major:0 minor:213 fsType:overlay blockSize:0} overlay_0-219:{mountpoint:/var/lib/containers/storage/overlay/58f23141b3f86bac09cb6959cd6b8c84198ef3910a151372aaa6920770b79e40/merged major:0 minor:219 fsType:overlay blockSize:0} overlay_0-223:{mountpoint:/var/lib/containers/storage/overlay/3a9e42182ccb946a4c410017fdd06fc596b96bbadf25578786bd8d6a951032e6/merged major:0 minor:223 fsType:overlay blockSize:0} overlay_0-225:{mountpoint:/var/lib/containers/storage/overlay/53ec7c97a8831a6aee8859e6eea2331b0a048fa0edf8dc32868ebfe7d05e8fb2/merged major:0 minor:225 fsType:overlay blockSize:0} overlay_0-227:{mountpoint:/var/lib/containers/storage/overlay/373233326df939daa80ce0e1807b51f3c3c8b764007be0b07bb8ff6ce41361ef/merged major:0 minor:227 fsType:overlay blockSize:0} overlay_0-229:{mountpoint:/var/lib/containers/storage/overlay/5b51489d4329b78a7e53a8324129a189ec262899eb7441b56f93db7718fe262a/merged major:0 minor:229 fsType:overlay blockSize:0} overlay_0-235:{mountpoint:/var/lib/containers/storage/overlay/8c74ddb156118e9c82e694a22df2d68e5e8144c5bf9ac1bcd4c79c6013b45f86/merged major:0 minor:235 fsType:overlay blockSize:0} overlay_0-241:{mountpoint:/var/lib/containers/storage/overlay/d4d1648a8f15272af7782383635776643ddd0c8ca194e5f67e7578a6dee19eb5/merged major:0 minor:241 fsType:overlay blockSize:0} overlay_0-247:{mountpoint:/var/lib/containers/storage/overlay/8ba186c2b1dc35bf0b02eb7f5e1414657faad775adff91f8f43d1f4fd3e29688/merged major:0 minor:247 fsType:overlay blockSize:0} overlay_0-51:{mountpoint:/var/lib/containers/storage/overlay/175fd28aec753d7a2d45d55f79b5199a8b12fde9ede72ba3e18475f30f17c8eb/merged major:0 minor:51 fsType:overlay blockSize:0} overlay_0-53:{mountpoint:/var/lib/containers/storage/overlay/6436a6d56abf29fb403b9ae82c90ff298ad1937adcede1bd60ee336b58a73251/merged major:0 minor:53 fsType:overlay blockSize:0} overlay_0-70:{mountpoint:/var/lib/containers/storage/overlay/7b15d6ba27c69007cfe88dc608e86358f1e7b266eb0fc4fd8c86987999f93abf/merged major:0 minor:70 fsType:overlay blockSize:0} overlay_0-74:{mountpoint:/var/lib/containers/storage/overlay/b4afc2129b478a63b8dc4abc9e1e0b170a2ba9ccc39620cee4e62306a1b07684/merged major:0 minor:74 fsType:overlay blockSize:0} overlay_0-78:{mountpoint:/var/lib/containers/storage/overlay/4da3f735e1f26a9fdb02a8fd82796ac7adc3a8b256930f9178a12d7957f51a6b/merged major:0 minor:78 fsType:overlay blockSize:0} overlay_0-82:{mountpoint:/var/lib/containers/storage/overlay/2578467e9cdcc49d7ad7e669d92e008be666e2a4463f8b18f23dde350bf88d33/merged major:0 minor:82 fsType:overlay blockSize:0} overlay_0-84:{mountpoint:/var/lib/containers/storage/overlay/287cab86ff088742d6baf629e0760ab3df45c2104663df5b342967898b2108b7/merged major:0 minor:84 fsType:overlay blockSize:0} overlay_0-88:{mountpoint:/var/lib/containers/storage/overlay/ea47366c6895fcac0a556b135bf4617f2b037d9d131125da116711dd55a612cf/merged major:0 minor:88 fsType:overlay blockSize:0} overlay_0-92:{mountpoint:/var/lib/containers/storage/overlay/ddf6bbc752431c8c1f637bb9e8bd2e348ee21beeb767a3f68351898b250de453/merged major:0 minor:92 fsType:overlay blockSize:0} overlay_0-97:{mountpoint:/var/lib/containers/storage/overlay/d5fa35a00b38b90bbbc8eea535d79736adb5f0f3bbf938183ddad5cae8f69c9f/merged major:0 minor:97 fsType:overlay blockSize:0}]
Oct 30 21:42:34.306736 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.304092    4473 manager.go:217] Machine: {Timestamp:2026-10-30 21:42:34.30360973 +0000 UTC m=+0.076978208 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3093462 MemoryCapacity:32812015616 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2f47f72d295f8172bcce343315896e SystemUUID:ec2f47f7-2d29-5f81-72bc-ce343315896e BootID:ee1b8de9-2692-4004-9fb2-0c9e0790d8fd Filesystems:[{Device:overlay_0-119 DeviceMajor:0 DeviceMinor:119 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-132 DeviceMajor:0 DeviceMinor:132 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/801bef426e6476fee21829f7a500535c139dc4114bb4ef9843db0c96de15297d/userdata/shm DeviceMajor:0 DeviceMinor:47 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-102 DeviceMajor:0 DeviceMinor:102 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-134 DeviceMajor:0 DeviceMinor:134 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/76411d00e5efeac0f8e2cdc0c51751cb906f91e91e8b427c7cd78c85f25ebfa6/userdata/shm DeviceMajor:0 DeviceMinor:221 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-229 DeviceMajor:0 DeviceMinor:229 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/06552eb8-7379-40df-b31c-a133e646afb9/volumes/kubernetes.io~secret/agent-certs DeviceMajor:0 DeviceMinor:58 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/27f3fa2df241ed15c0c679361f882d1b7a1c504e1f510e4d568952e3f43917d3/userdata/shm DeviceMajor:0 DeviceMinor:72 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/f152360c-cf62-4d10-b2e5-b85ac7dfb739/volumes/kubernetes.io~projected/kube-api-access-n6qcb DeviceMajor:0 DeviceMinor:201 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/5e9d9d5b697e87a3519595619e275bc9ad494e247a8e7e1892ea4d63b50a4a56/userdata/shm DeviceMajor:0 DeviceMinor:207 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-74 DeviceMajor:0 DeviceMinor:74 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-139 DeviceMajor:0 DeviceMinor:139 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-162 DeviceMajor:0 DeviceMinor:162 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/4436ec50-e190-4c1c-8b6c-2b9b5d12cb9f/volumes/kubernetes.io~secret/cert DeviceMajor:0 DeviceMinor:45 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-167 DeviceMajor:0 DeviceMinor:167 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-143 DeviceMajor:0 DeviceMinor:143 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-51 DeviceMajor:0 DeviceMinor:51 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/c67fe06a-8f45-48d0-9451-b9157f57369e/volumes/kubernetes.io~empty-dir/tmp DeviceMajor:0 DeviceMinor:55 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/5de0b6c7-0d1c-4732-abce-361a46df75c3/volumes/kubernetes.io~projected/kube-api-access-rfxjl DeviceMajor:0 DeviceMinor:60 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/ff246766b2304f22246fdd270a6b38453bc6d295d4e10b3536995b636b482eae/userdata/shm DeviceMajor:0 DeviceMinor:94 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/20fb5e527b7a589fc2087d3651238f91f4c7354431ea9ec95b08f8d6e78d96c6/userdata/shm DeviceMajor:0 DeviceMinor:95 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-97 DeviceMajor:0 DeviceMinor:97 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6562406400 Type:vfs Inodes:819200 HasInodes:true} {Device:overlay_0-53 DeviceMajor:0 DeviceMinor:53 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/2428ae10-7385-4056-aabe-bdf1b974a36b/volumes/kubernetes.io~projected/kube-api-access-5xlxd DeviceMajor:0 DeviceMinor:66 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/0743c782-f5ad-4d3a-86e7-fc66563902c7/volumes/kubernetes.io~projected/kube-api-access-r9bdf DeviceMajor:0 DeviceMinor:67 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/79db91fb4dc34e8d30ddec93a6117fc1d4443038b0f2d8fc5c72483c4e7c1e68/userdata/shm DeviceMajor:0 DeviceMinor:80 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-92 DeviceMajor:0 DeviceMinor:92 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/4436ec50-e190-4c1c-8b6c-2b9b5d12cb9f/volumes/kubernetes.io~projected/kube-api-access-7vks4 DeviceMajor:0 DeviceMinor:197 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/bddede0e-ee07-4344-878d-10b967435e13/volumes/kubernetes.io~projected/kube-api-access-lnn4t DeviceMajor:0 DeviceMinor:216 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-84 DeviceMajor:0 DeviceMinor:84 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/861ed8ec8169a70ec732075a3a81d7cce667a723e813db6aa2087d0be3b3bcf5/userdata/shm DeviceMajor:0 DeviceMinor:86 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-88 DeviceMajor:0 DeviceMinor:88 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-108 DeviceMajor:0 DeviceMinor:108 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-172 DeviceMajor:0 DeviceMinor:172 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/var/lib/kubelet/pods/03a62eeb-c318-4ce3-b6e9-1afba98920e9/volumes/kubernetes.io~projected/kube-api-access-9bq8j DeviceMajor:0 DeviceMinor:63 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/719f98aa-6247-4973-8d2c-a91aaf276984/volumes/kubernetes.io~projected/kube-api-access-pn9mk DeviceMajor:0 DeviceMinor:65 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-78 DeviceMajor:0 DeviceMinor:78 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-157 DeviceMajor:0 DeviceMinor:157 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/10d2e5ae-c1c4-400c-8c67-3aebb6caaa6a/volumes/kubernetes.io~projected/kube-api-access-js6zb DeviceMajor:0 DeviceMinor:209 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-247 DeviceMajor:0 DeviceMinor:247 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-104 DeviceMajor:0 DeviceMinor:104 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-225 DeviceMajor:0 DeviceMinor:225 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-241 DeviceMajor:0 DeviceMinor:241 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/c829b82f78e3de9b1b8ccab3a669062ac59ed7b3e800a2ef83c3037dcc6b0b75/userdata/shm DeviceMajor:0 DeviceMinor:48 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/7b5fc3aef820c3083e5409913ee1bc84b01ce70c5542392f5f23ae7c3d241c67/userdata/shm DeviceMajor:0 DeviceMinor:90 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-192 DeviceMajor:0 DeviceMinor:192 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-213 DeviceMajor:0 DeviceMinor:213 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16406007808 Type:vfs Inodes:1048576 HasInodes:true} {Device:/var/lib/kubelet/pods/ccc5af13-906a-469d-9ca5-09366d1a7e93/volumes/kubernetes.io~projected/kube-api-access-b8x4j DeviceMajor:0 DeviceMinor:59 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/2634eb5b-4ce8-4742-819a-616dcd33da39/volumes/kubernetes.io~projected/kube-api-access-htnvw DeviceMajor:0 DeviceMinor:62 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-105 DeviceMajor:0 DeviceMinor:105 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-124 DeviceMajor:0 DeviceMinor:124 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/7a14b2934bbe1c4ee21f714f16eb03e32018d69985a144c98a7450961a811688/userdata/shm DeviceMajor:0 DeviceMinor:202 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-204 DeviceMajor:0 DeviceMinor:204 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/c30e39faca61472a40248b9cb2c2aecc5cd89074cddd409ced6aa73a7e97ff49/userdata/shm DeviceMajor:0 DeviceMinor:211 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-182 DeviceMajor:0 DeviceMinor:182 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-227 DeviceMajor:0 DeviceMinor:227 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-100 DeviceMajor:0 DeviceMinor:100 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/var/lib/kubelet/pods/c67fe06a-8f45-48d0-9451-b9157f57369e/volumes/kubernetes.io~empty-dir/etc-tuned DeviceMajor:0 DeviceMinor:56 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/641ac65e-00db-4646-b03c-9e3cb0c70feb/volume-subpaths/run-systemd/ovnkube-controller/6 DeviceMajor:0 DeviceMinor:26 Capacity:6562406400 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16406007808 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-70 DeviceMajor:0 DeviceMinor:70 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-177 DeviceMajor:0 DeviceMinor:177 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/525a0437-9a44-425b-966a-b118d91a4522/volumes/kubernetes.io~projected/kube-api-access-g98k9 DeviceMajor:0 DeviceMinor:215 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/ea9ef8f120528befa45af4e9c5d895ac4de2f82aa8e6d58db49358916aa15775/userdata/shm DeviceMajor:0 DeviceMinor:217 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-219 DeviceMajor:0 DeviceMinor:219 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-82 DeviceMajor:0 DeviceMinor:82 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-127 DeviceMajor:0 DeviceMinor:127 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-152 DeviceMajor:0 DeviceMinor:152 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-187 DeviceMajor:0 DeviceMinor:187 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-223 DeviceMajor:0 DeviceMinor:223 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-235 DeviceMajor:0 DeviceMinor:235 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/641ac65e-00db-4646-b03c-9e3cb0c70feb/volumes/kubernetes.io~projected/kube-api-access-9hm6j DeviceMajor:0 DeviceMinor:64 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/47636ce1c9aefb50eea1cbdaf7ddbc964ffdd48cb1b0c4160ebe0fc9c9f929e9/userdata/shm DeviceMajor:0 DeviceMinor:68 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/d8fa13f6ee6c04e40723c9b7bd7388ae0cb58ff50531dfd0e89bf94ad5309b77/userdata/shm DeviceMajor:0 DeviceMinor:76 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/0032cb99-3a37-40c6-98cb-2d40ae4a9319/volumes/kubernetes.io~secret/original-pull-secret DeviceMajor:0 DeviceMinor:206 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/32a1c660-effa-4b6c-975a-dbb68c468eda/volumes/kubernetes.io~projected/kube-api-access-zzqrt DeviceMajor:0 DeviceMinor:210 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/641ac65e-00db-4646-b03c-9e3cb0c70feb/volumes/kubernetes.io~secret/ovn-node-metrics-cert DeviceMajor:0 DeviceMinor:57 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/c67fe06a-8f45-48d0-9451-b9157f57369e/volumes/kubernetes.io~projected/kube-api-access-64s8x DeviceMajor:0 DeviceMinor:61 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:5e9d9d5b697e87a MacAddress:0a:f0:31:eb:ff:0a Speed:10000 Mtu:8901} {Name:76411d00e5efeac MacAddress:e2:68:b8:a7:27:08 Speed:10000 Mtu:8901} {Name:7a14b2934bbe1c4 MacAddress:f2:60:05:03:77:7b Speed:10000 Mtu:8901} {Name:br-ex MacAddress:02:5d:e4:f6:29:1f Speed:0 Mtu:9001} {Name:br-int MacAddress:02:17:7a:9b:0e:62 Speed:0 Mtu:8901} {Name:c30e39faca61472 MacAddress:9a:12:a8:90:d7:a1 Speed:10000 Mtu:8901} {Name:ea9ef8f120528be MacAddress:3e:82:19:6d:4c:ab Speed:10000 Mtu:8901} {Name:ens5 MacAddress:02:5d:e4:f6:29:1f Speed:0 Mtu:9001} {Name:genev_sys_6081 MacAddress:36:66:39:e6:32:b0 Speed:0 Mtu:65000} {Name:ovn-k8s-mp0 MacAddress:0a:58:0a:85:00:02 Speed:0 Mtu:8901} {Name:ovs-system MacAddress:fa:08:63:32:2d:6a Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812015616 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Oct 30 21:42:34.306736 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.304353    4473 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Oct 30 21:42:34.306736 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.304403    4473 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Oct 30 21:42:34.306736 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.304660    4473 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Oct 30 21:42:34.306736 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.304678    4473 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-140-18.us-west-1.compute.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Oct 30 21:42:34.306736 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.304813    4473 topology_manager.go:143] "Creating topology manager with none policy"
Oct 30 21:42:34.306736 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.304820    4473 container_manager_linux.go:311] "Creating device plugin manager"
Oct 30 21:42:34.306736 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.304833    4473 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 30 21:42:34.306736 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.304852    4473 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 30 21:42:34.306736 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.304960    4473 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager"
Oct 30 21:42:34.306736 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.305143    4473 state_mem.go:41] "Initialized" logger="CPUManager state memory"
Oct 30 21:42:34.306736 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.305257    4473 server.go:1235] "Using root directory" path="/var/lib/kubelet"
Oct 30 21:42:34.306736 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.305292    4473 kubelet.go:493] "Attempting to sync node with API server"
Oct 30 21:42:34.306736 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.305302    4473 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests"
Oct 30 21:42:34.306736 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.305312    4473 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Oct 30 21:42:34.306736 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.305321    4473 kubelet.go:405] "Adding apiserver pod source"
Oct 30 21:42:34.306736 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.305330    4473 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Oct 30 21:42:34.306736 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.305978    4473 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint"
Oct 30 21:42:34.306736 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.305990    4473 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 30 21:42:34.306736 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.306566    4473 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1"
Oct 30 21:42:34.308468 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.308454    4473 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint"
Oct 30 21:42:34.308468 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.308469    4473 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 30 21:42:34.308575 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.308566    4473 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-server-current.pem"
Oct 30 21:42:34.308766 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.308750    4473 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Oct 30 21:42:34.308800 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.308785    4473 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Oct 30 21:42:34.308881 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.308873    4473 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Oct 30 21:42:34.308910 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.308887    4473 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Oct 30 21:42:34.308910 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.308900    4473 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Oct 30 21:42:34.308910 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.308905    4473 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Oct 30 21:42:34.309062 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.308912    4473 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Oct 30 21:42:34.309062 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.308936    4473 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Oct 30 21:42:34.309062 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.308943    4473 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Oct 30 21:42:34.309062 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.308949    4473 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Oct 30 21:42:34.309062 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.308955    4473 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Oct 30 21:42:34.309062 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.308961    4473 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Oct 30 21:42:34.309062 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.308974    4473 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Oct 30 21:42:34.309062 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.308983    4473 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Oct 30 21:42:34.309062 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.309006    4473 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Oct 30 21:42:34.309062 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.309012    4473 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Oct 30 21:42:34.312315 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.312302    4473 server.go:1263] "Started kubelet"
Oct 30 21:42:34.312413 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.312383    4473 server.go:182] "Starting to listen" address="0.0.0.0" port=10250
Oct 30 21:42:34.312850 ip-10-0-140-18 systemd[1]: Started Kubernetes Kubelet.
Oct 30 21:42:34.313070 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.312873    4473 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Oct 30 21:42:34.313070 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.312974    4473 server_v1.go:49] "podresources" method="list" useActivePods=true
Oct 30 21:42:34.313300 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.313279    4473 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Oct 30 21:42:34.320441 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.315459    4473 server.go:317] "Adding debug handlers to kubelet server"
Oct 30 21:42:34.320441 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.317295    4473 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer"
Oct 30 21:42:34.320441 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.317304    4473 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Oct 30 21:42:34.320441 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.317522    4473 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-29 21:36:55 +0000 UTC" deadline="2028-07-31 01:14:10.781333526 +0000 UTC"
Oct 30 21:42:34.320441 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.317563    4473 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15339h31m36.463780251s"
Oct 30 21:42:34.320441 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.317666    4473 volume_manager.go:309] "The desired_state_of_world populator starts"
Oct 30 21:42:34.320441 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.317695    4473 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Oct 30 21:42:34.320441 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.317852    4473 volume_manager.go:311] "Starting Kubelet Volume Manager"
Oct 30 21:42:34.320441 ip-10-0-140-18 kubenswrapper[4473]: E1030 21:42:34.317703    4473 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-140-18.us-west-1.compute.internal\" not found"
Oct 30 21:42:34.320441 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.320212    4473 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2634eb5b-4ce8-4742-819a-616dcd33da39" volumeName="kubernetes.io/projected/2634eb5b-4ce8-4742-819a-616dcd33da39-kube-api-access-htnvw" seLinuxMountContext=""
Oct 30 21:42:34.320441 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.320237    4473 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="5de0b6c7-0d1c-4732-abce-361a46df75c3" volumeName="kubernetes.io/projected/5de0b6c7-0d1c-4732-abce-361a46df75c3-kube-api-access-rfxjl" seLinuxMountContext=""
Oct 30 21:42:34.320441 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.320247    4473 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="641ac65e-00db-4646-b03c-9e3cb0c70feb" volumeName="kubernetes.io/projected/641ac65e-00db-4646-b03c-9e3cb0c70feb-kube-api-access-9hm6j" seLinuxMountContext=""
Oct 30 21:42:34.320441 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.320255    4473 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="641ac65e-00db-4646-b03c-9e3cb0c70feb" volumeName="kubernetes.io/secret/641ac65e-00db-4646-b03c-9e3cb0c70feb-ovn-node-metrics-cert" seLinuxMountContext=""
Oct 30 21:42:34.320441 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.320263    4473 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2428ae10-7385-4056-aabe-bdf1b974a36b" volumeName="kubernetes.io/projected/2428ae10-7385-4056-aabe-bdf1b974a36b-kube-api-access-5xlxd" seLinuxMountContext=""
Oct 30 21:42:34.320441 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.320287    4473 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0032cb99-3a37-40c6-98cb-2d40ae4a9319" volumeName="kubernetes.io/secret/0032cb99-3a37-40c6-98cb-2d40ae4a9319-original-pull-secret" seLinuxMountContext=""
Oct 30 21:42:34.320441 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.320295    4473 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="10d2e5ae-c1c4-400c-8c67-3aebb6caaa6a" volumeName="kubernetes.io/projected/10d2e5ae-c1c4-400c-8c67-3aebb6caaa6a-kube-api-access-js6zb" seLinuxMountContext=""
Oct 30 21:42:34.320441 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.320304    4473 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="32a1c660-effa-4b6c-975a-dbb68c468eda" volumeName="kubernetes.io/projected/32a1c660-effa-4b6c-975a-dbb68c468eda-kube-api-access-zzqrt" seLinuxMountContext=""
Oct 30 21:42:34.320441 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.320313    4473 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="719f98aa-6247-4973-8d2c-a91aaf276984" volumeName="kubernetes.io/configmap/719f98aa-6247-4973-8d2c-a91aaf276984-iptables-alerter-script" seLinuxMountContext=""
Oct 30 21:42:34.320441 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.320320    4473 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="06552eb8-7379-40df-b31c-a133e646afb9" volumeName="kubernetes.io/secret/06552eb8-7379-40df-b31c-a133e646afb9-agent-certs" seLinuxMountContext=""
Oct 30 21:42:34.320441 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.320327    4473 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0743c782-f5ad-4d3a-86e7-fc66563902c7" volumeName="kubernetes.io/projected/0743c782-f5ad-4d3a-86e7-fc66563902c7-kube-api-access-r9bdf" seLinuxMountContext=""
Oct 30 21:42:34.320441 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.320334    4473 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="03a62eeb-c318-4ce3-b6e9-1afba98920e9" volumeName="kubernetes.io/projected/03a62eeb-c318-4ce3-b6e9-1afba98920e9-kube-api-access-9bq8j" seLinuxMountContext=""
Oct 30 21:42:34.320441 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.320340    4473 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="06552eb8-7379-40df-b31c-a133e646afb9" volumeName="kubernetes.io/configmap/06552eb8-7379-40df-b31c-a133e646afb9-konnectivity-ca" seLinuxMountContext=""
Oct 30 21:42:34.320441 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.320347    4473 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0743c782-f5ad-4d3a-86e7-fc66563902c7" volumeName="kubernetes.io/configmap/0743c782-f5ad-4d3a-86e7-fc66563902c7-cni-sysctl-allowlist" seLinuxMountContext=""
Oct 30 21:42:34.320441 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.320361    4473 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="03a62eeb-c318-4ce3-b6e9-1afba98920e9" volumeName="kubernetes.io/configmap/03a62eeb-c318-4ce3-b6e9-1afba98920e9-multus-daemon-config" seLinuxMountContext=""
Oct 30 21:42:34.320441 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.320368    4473 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="641ac65e-00db-4646-b03c-9e3cb0c70feb" volumeName="kubernetes.io/configmap/641ac65e-00db-4646-b03c-9e3cb0c70feb-ovnkube-script-lib" seLinuxMountContext=""
Oct 30 21:42:34.320441 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.320375    4473 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="719f98aa-6247-4973-8d2c-a91aaf276984" volumeName="kubernetes.io/projected/719f98aa-6247-4973-8d2c-a91aaf276984-kube-api-access-pn9mk" seLinuxMountContext=""
Oct 30 21:42:34.320441 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.320382    4473 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bddede0e-ee07-4344-878d-10b967435e13" volumeName="kubernetes.io/configmap/bddede0e-ee07-4344-878d-10b967435e13-trusted-ca-bundle" seLinuxMountContext=""
Oct 30 21:42:34.320441 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.320388    4473 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bddede0e-ee07-4344-878d-10b967435e13" volumeName="kubernetes.io/empty-dir/bddede0e-ee07-4344-878d-10b967435e13-archives-path" seLinuxMountContext=""
Oct 30 21:42:34.320441 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.320395    4473 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bddede0e-ee07-4344-878d-10b967435e13" volumeName="kubernetes.io/projected/bddede0e-ee07-4344-878d-10b967435e13-kube-api-access-lnn4t" seLinuxMountContext=""
Oct 30 21:42:34.320441 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.320401    4473 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c67fe06a-8f45-48d0-9451-b9157f57369e" volumeName="kubernetes.io/empty-dir/c67fe06a-8f45-48d0-9451-b9157f57369e-tmp" seLinuxMountContext=""
Oct 30 21:42:34.320441 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.320408    4473 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c67fe06a-8f45-48d0-9451-b9157f57369e" volumeName="kubernetes.io/empty-dir/c67fe06a-8f45-48d0-9451-b9157f57369e-etc-tuned" seLinuxMountContext=""
Oct 30 21:42:34.320441 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.320415    4473 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0743c782-f5ad-4d3a-86e7-fc66563902c7" volumeName="kubernetes.io/configmap/0743c782-f5ad-4d3a-86e7-fc66563902c7-cni-binary-copy" seLinuxMountContext=""
Oct 30 21:42:34.320441 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.320422    4473 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="32a1c660-effa-4b6c-975a-dbb68c468eda" volumeName="kubernetes.io/empty-dir/32a1c660-effa-4b6c-975a-dbb68c468eda-data-volume" seLinuxMountContext=""
Oct 30 21:42:34.320441 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.320428    4473 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4436ec50-e190-4c1c-8b6c-2b9b5d12cb9f" volumeName="kubernetes.io/secret/4436ec50-e190-4c1c-8b6c-2b9b5d12cb9f-cert" seLinuxMountContext=""
Oct 30 21:42:34.320441 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.320434    4473 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="525a0437-9a44-425b-966a-b118d91a4522" volumeName="kubernetes.io/projected/525a0437-9a44-425b-966a-b118d91a4522-kube-api-access-g98k9" seLinuxMountContext=""
Oct 30 21:42:34.320441 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.320440    4473 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f152360c-cf62-4d10-b2e5-b85ac7dfb739" volumeName="kubernetes.io/empty-dir/f152360c-cf62-4d10-b2e5-b85ac7dfb739-tmp-dir" seLinuxMountContext=""
Oct 30 21:42:34.320441 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.320446    4473 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f152360c-cf62-4d10-b2e5-b85ac7dfb739" volumeName="kubernetes.io/projected/f152360c-cf62-4d10-b2e5-b85ac7dfb739-kube-api-access-n6qcb" seLinuxMountContext=""
Oct 30 21:42:34.320441 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.320456    4473 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bddede0e-ee07-4344-878d-10b967435e13" volumeName="kubernetes.io/configmap/bddede0e-ee07-4344-878d-10b967435e13-service-ca-bundle" seLinuxMountContext=""
Oct 30 21:42:34.320441 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.320463    4473 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="03a62eeb-c318-4ce3-b6e9-1afba98920e9" volumeName="kubernetes.io/configmap/03a62eeb-c318-4ce3-b6e9-1afba98920e9-cni-binary-copy" seLinuxMountContext=""
Oct 30 21:42:34.321572 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.320471    4473 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0743c782-f5ad-4d3a-86e7-fc66563902c7" volumeName="kubernetes.io/configmap/0743c782-f5ad-4d3a-86e7-fc66563902c7-whereabouts-flatfile-configmap" seLinuxMountContext=""
Oct 30 21:42:34.321572 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.320477    4473 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4436ec50-e190-4c1c-8b6c-2b9b5d12cb9f" volumeName="kubernetes.io/projected/4436ec50-e190-4c1c-8b6c-2b9b5d12cb9f-kube-api-access-7vks4" seLinuxMountContext=""
Oct 30 21:42:34.321572 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.320484    4473 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="641ac65e-00db-4646-b03c-9e3cb0c70feb" volumeName="kubernetes.io/configmap/641ac65e-00db-4646-b03c-9e3cb0c70feb-ovnkube-config" seLinuxMountContext=""
Oct 30 21:42:34.321572 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.320490    4473 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="641ac65e-00db-4646-b03c-9e3cb0c70feb" volumeName="kubernetes.io/configmap/641ac65e-00db-4646-b03c-9e3cb0c70feb-env-overrides" seLinuxMountContext=""
Oct 30 21:42:34.321572 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.320497    4473 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c67fe06a-8f45-48d0-9451-b9157f57369e" volumeName="kubernetes.io/projected/c67fe06a-8f45-48d0-9451-b9157f57369e-kube-api-access-64s8x" seLinuxMountContext=""
Oct 30 21:42:34.321572 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.320504    4473 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ccc5af13-906a-469d-9ca5-09366d1a7e93" volumeName="kubernetes.io/projected/ccc5af13-906a-469d-9ca5-09366d1a7e93-kube-api-access-b8x4j" seLinuxMountContext=""
Oct 30 21:42:34.321572 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.320510    4473 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f152360c-cf62-4d10-b2e5-b85ac7dfb739" volumeName="kubernetes.io/configmap/f152360c-cf62-4d10-b2e5-b85ac7dfb739-config-volume" seLinuxMountContext=""
Oct 30 21:42:34.321572 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.320517    4473 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2428ae10-7385-4056-aabe-bdf1b974a36b" volumeName="kubernetes.io/configmap/2428ae10-7385-4056-aabe-bdf1b974a36b-serviceca" seLinuxMountContext=""
Oct 30 21:42:34.321572 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.320527    4473 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2634eb5b-4ce8-4742-819a-616dcd33da39" volumeName="kubernetes.io/empty-dir/2634eb5b-4ce8-4742-819a-616dcd33da39-tmp-dir" seLinuxMountContext=""
Oct 30 21:42:34.321572 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.320534    4473 reconstruct.go:97] "Volume reconstruction finished"
Oct 30 21:42:34.321572 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.320539    4473 reconciler.go:29] "Reconciler: start to sync state"
Oct 30 21:42:34.322014 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.321899    4473 factory.go:55] Registering systemd factory
Oct 30 21:42:34.322014 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.321933    4473 factory.go:223] Registration of the systemd container factory successfully
Oct 30 21:42:34.324265 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.324246    4473 factory.go:153] Registering CRI-O factory
Oct 30 21:42:34.324265 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.324266    4473 factory.go:223] Registration of the crio container factory successfully
Oct 30 21:42:34.324408 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.324320    4473 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Oct 30 21:42:34.324408 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.324343    4473 factory.go:103] Registering Raw factory
Oct 30 21:42:34.324408 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.324356    4473 manager.go:1199] Started watching for new ooms in manager
Oct 30 21:42:34.324636 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.324626    4473 manager.go:319] Starting recovery of all containers
Oct 30 21:42:34.347565 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.347526    4473 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Oct 30 21:42:34.349015 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.348986    4473 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Oct 30 21:42:34.349015 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.349009    4473 status_manager.go:249] "Starting to sync pod status with apiserver"
Oct 30 21:42:34.349174 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.349030    4473 kubelet.go:2521] "Starting kubelet main sync loop"
Oct 30 21:42:34.349174 ip-10-0-140-18 kubenswrapper[4473]: E1030 21:42:34.349062    4473 kubelet.go:2545] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]"
Oct 30 21:42:34.370846 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.370807    4473 generic.go:356] "Generic (PLEG): container finished" podID="9a45018bbca93ea1080adf030d8fbb7f" containerID="655da74309b94247c02b665d6dd5306bf1c511226bfa3b60103fc150690d1d80" exitCode=0
Oct 30 21:42:34.379638 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.379608    4473 generic.go:356] "Generic (PLEG): container finished" podID="0743c782-f5ad-4d3a-86e7-fc66563902c7" containerID="0a7714b7c94a15bd15123619d730d6d97adea79a55948f28db83c43b8b9ad43e" exitCode=0
Oct 30 21:42:34.379638 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.379637    4473 generic.go:356] "Generic (PLEG): container finished" podID="0743c782-f5ad-4d3a-86e7-fc66563902c7" containerID="17b069cd8fc7a446971e3d4c429d877a859dc5d6e87ca3ad7826fed74e833bc6" exitCode=0
Oct 30 21:42:34.379818 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.379645    4473 generic.go:356] "Generic (PLEG): container finished" podID="0743c782-f5ad-4d3a-86e7-fc66563902c7" containerID="0471160a86a336a9b462e092eee66ccd343b1e11033691c6220597d8aa122a3c" exitCode=0
Oct 30 21:42:34.379818 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.379655    4473 generic.go:356] "Generic (PLEG): container finished" podID="0743c782-f5ad-4d3a-86e7-fc66563902c7" containerID="31add3ceb278330ecedcba57b30a664d06a40dc12c8d453941a66cf981d33c4e" exitCode=0
Oct 30 21:42:34.379818 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.379663    4473 generic.go:356] "Generic (PLEG): container finished" podID="0743c782-f5ad-4d3a-86e7-fc66563902c7" containerID="1b8b63f9328443c9ce89909435c5e73442ca77b64a8fd94d708b5b2c192353a5" exitCode=0
Oct 30 21:42:34.379818 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.379672    4473 generic.go:356] "Generic (PLEG): container finished" podID="0743c782-f5ad-4d3a-86e7-fc66563902c7" containerID="e1b3266e356dd8341142ab24bc0f3fc3760ac6bf9b7520077225109599c6a4e6" exitCode=0
Oct 30 21:42:34.394087 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.394062    4473 manager.go:324] Recovery completed
Oct 30 21:42:34.407256 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.407233    4473 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 21:42:34.408416 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.408398    4473 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-18.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Oct 30 21:42:34.408498 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.408430    4473 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-18.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 21:42:34.408498 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.408442    4473 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-18.us-west-1.compute.internal" event="NodeHasSufficientPID"
Oct 30 21:42:34.409931 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.409901    4473 cpu_manager.go:226] "Starting" policy="none"
Oct 30 21:42:34.410000 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.409931    4473 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 21:42:34.410030 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.409932    4473 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s"
Oct 30 21:42:34.410070 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.410027    4473 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory"
Oct 30 21:42:34.410188 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.410163    4473 state_mem.go:94] "Updated default CPUSet" logger="CPUManager state checkpoint.CPUManager state memory" cpuSet=""
Oct 30 21:42:34.410241 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.410178    4473 state_mem.go:102] "Updated CPUSet assignments" logger="CPUManager state checkpoint.CPUManager state memory" assignments={}
Oct 30 21:42:34.410241 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.410201    4473 state_checkpoint.go:142] "restored state from checkpoint" logger="CPUManager state checkpoint"
Oct 30 21:42:34.410241 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.410209    4473 state_checkpoint.go:143] "defaultCPUSet" logger="CPUManager state checkpoint" defaultCpuSet=""
Oct 30 21:42:34.410241 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.410217    4473 policy_none.go:50] "Start"
Oct 30 21:42:34.410241 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.410230    4473 memory_manager.go:187] "Starting memorymanager" policy="None"
Oct 30 21:42:34.410241 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.410242    4473 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Oct 30 21:42:34.410399 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.410375    4473 state_mem.go:77] "Updated machine memory state" logger="Memory Manager state checkpoint"
Oct 30 21:42:34.410399 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.410383    4473 state_checkpoint.go:85] "State checkpoint: restored state from checkpoint" logger="Memory Manager state checkpoint"
Oct 30 21:42:34.410399 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.410388    4473 policy_none.go:44] "Start"
Oct 30 21:42:34.410777 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.410763    4473 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-18.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Oct 30 21:42:34.410821 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.410795    4473 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-18.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 21:42:34.410821 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.410808    4473 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-18.us-west-1.compute.internal" event="NodeHasSufficientPID"
Oct 30 21:42:34.415751 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.415717    4473 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"}
Oct 30 21:42:34.418489 ip-10-0-140-18 kubenswrapper[4473]: E1030 21:42:34.418439    4473 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-140-18.us-west-1.compute.internal\" not found"
Oct 30 21:42:34.418940 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.418911    4473 manager.go:341] "Starting Device Plugin manager"
Oct 30 21:42:34.418990 ip-10-0-140-18 kubenswrapper[4473]: E1030 21:42:34.418956    4473 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Oct 30 21:42:34.418990 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.418969    4473 server.go:87] "Starting device plugin registration server"
Oct 30 21:42:34.419233 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.419220    4473 eviction_manager.go:194] "Eviction manager: starting control loop"
Oct 30 21:42:34.419300 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.419233    4473 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Oct 30 21:42:34.419346 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.419312    4473 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Oct 30 21:42:34.419449 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.419404    4473 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Oct 30 21:42:34.419519 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.419463    4473 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts"
Oct 30 21:42:34.419519 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.419471    4473 plugin_manager.go:121] "Starting Kubelet Plugin Manager"
Oct 30 21:42:34.419602 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.419535    4473 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-10-30T21:42:34.419450042Z","UUID":"47b65c2b-d1a1-4e65-9a72-93996647cd70","Handler":null,"Name":"","Endpoint":""}
Oct 30 21:42:34.420669 ip-10-0-140-18 kubenswrapper[4473]: E1030 21:42:34.420644    4473 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Oct 30 21:42:34.420776 ip-10-0-140-18 kubenswrapper[4473]: E1030 21:42:34.420684    4473 eviction_manager.go:297] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-140-18.us-west-1.compute.internal\" not found"
Oct 30 21:42:34.420910 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.420894    4473 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Oct 30 21:42:34.420969 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.420946    4473 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Oct 30 21:42:34.450104 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.449593    4473 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-18.us-west-1.compute.internal","kube-system/kube-apiserver-proxy-ip-10-0-140-18.us-west-1.compute.internal"]
Oct 30 21:42:34.450257 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.450209    4473 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 21:42:34.451111 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.451094    4473 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-18.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Oct 30 21:42:34.451224 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.451125    4473 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-18.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 21:42:34.451224 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.451139    4473 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-18.us-west-1.compute.internal" event="NodeHasSufficientPID"
Oct 30 21:42:34.451448 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.451435    4473 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 21:42:34.451584 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.451566    4473 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 21:42:34.452174 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.452152    4473 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-18.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Oct 30 21:42:34.452174 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.452164    4473 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-18.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Oct 30 21:42:34.452336 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.452184    4473 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-18.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 21:42:34.452336 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.452185    4473 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-18.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 21:42:34.452336 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.452214    4473 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-18.us-west-1.compute.internal" event="NodeHasSufficientPID"
Oct 30 21:42:34.452336 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.452234    4473 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-18.us-west-1.compute.internal" event="NodeHasSufficientPID"
Oct 30 21:42:34.452514 ip-10-0-140-18 kubenswrapper[4473]: E1030 21:42:34.452379    4473 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-18.us-west-1.compute.internal\" not found" node="ip-10-0-140-18.us-west-1.compute.internal"
Oct 30 21:42:34.452583 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.452546    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-18.us-west-1.compute.internal" event={"ID":"9a45018bbca93ea1080adf030d8fbb7f","Type":"ContainerStarted","Data":"ea286be6562ac7caf78ebe558a341016bcd7592f2dc2b2262b6369d191be2da1"}
Oct 30 21:42:34.452644 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.452596    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-18.us-west-1.compute.internal" event={"ID":"9a45018bbca93ea1080adf030d8fbb7f","Type":"ContainerDied","Data":"655da74309b94247c02b665d6dd5306bf1c511226bfa3b60103fc150690d1d80"}
Oct 30 21:42:34.452644 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.452615    4473 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 21:42:34.452644 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.452621    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-18.us-west-1.compute.internal" event={"ID":"9a45018bbca93ea1080adf030d8fbb7f","Type":"ContainerStarted","Data":"801bef426e6476fee21829f7a500535c139dc4114bb4ef9843db0c96de15297d"}
Oct 30 21:42:34.452774 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.452678    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-18.us-west-1.compute.internal" event={"ID":"aae6ff4840f4ffd5842f280d0afa74f6","Type":"ContainerStarted","Data":"f94328bfc9f11e4b93eac895167a7f89cea18e245973618e123f60d14ca12d17"}
Oct 30 21:42:34.452774 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.452699    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-18.us-west-1.compute.internal" event={"ID":"aae6ff4840f4ffd5842f280d0afa74f6","Type":"ContainerStarted","Data":"c829b82f78e3de9b1b8ccab3a669062ac59ed7b3e800a2ef83c3037dcc6b0b75"}
Oct 30 21:42:34.453383 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.453366    4473 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-18.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Oct 30 21:42:34.453456 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.453398    4473 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-18.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 21:42:34.453456 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.453413    4473 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-18.us-west-1.compute.internal" event="NodeHasSufficientPID"
Oct 30 21:42:34.453552 ip-10-0-140-18 kubenswrapper[4473]: E1030 21:42:34.453541    4473 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-18.us-west-1.compute.internal\" not found" node="ip-10-0-140-18.us-west-1.compute.internal"
Oct 30 21:42:34.518503 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.518454    4473 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 21:42:34.518503 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.518491    4473 reconstruct.go:205] "DevicePaths of reconstructed volumes updated"
Oct 30 21:42:34.518765 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.518746    4473 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 21:42:34.519282 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.519256    4473 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-18.us-west-1.compute.internal"
Oct 30 21:42:34.521036 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.521022    4473 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 21:42:34.521992 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.521977    4473 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-18.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Oct 30 21:42:34.522090 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.522011    4473 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-18.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 21:42:34.522090 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.522025    4473 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-18.us-west-1.compute.internal" event="NodeHasSufficientPID"
Oct 30 21:42:34.522090 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.522069    4473 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-140-18.us-west-1.compute.internal"
Oct 30 21:42:34.569598 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.569566    4473 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 21:42:34.569733 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.569604    4473 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 21:42:34.614805 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.614770    4473 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 21:42:34.614964 ip-10-0-140-18 kubenswrapper[4473]: E1030 21:42:34.614860    4473 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-140-18.us-west-1.compute.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-18.us-west-1.compute.internal"
Oct 30 21:42:34.614964 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.614885    4473 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-18.us-west-1.compute.internal"
Oct 30 21:42:34.619087 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.619067    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/9a45018bbca93ea1080adf030d8fbb7f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-18.us-west-1.compute.internal\" (UID: \"9a45018bbca93ea1080adf030d8fbb7f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-18.us-west-1.compute.internal"
Oct 30 21:42:34.619151 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.619096    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9a45018bbca93ea1080adf030d8fbb7f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-18.us-west-1.compute.internal\" (UID: \"9a45018bbca93ea1080adf030d8fbb7f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-18.us-west-1.compute.internal"
Oct 30 21:42:34.619151 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.619115    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/aae6ff4840f4ffd5842f280d0afa74f6-config\") pod \"kube-apiserver-proxy-ip-10-0-140-18.us-west-1.compute.internal\" (UID: \"aae6ff4840f4ffd5842f280d0afa74f6\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-18.us-west-1.compute.internal"
Oct 30 21:42:34.667550 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.667523    4473 kubelet_node_status.go:126] "Node was previously registered" node="ip-10-0-140-18.us-west-1.compute.internal"
Oct 30 21:42:34.667708 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.667599    4473 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-140-18.us-west-1.compute.internal"
Oct 30 21:42:34.668237 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.668212    4473 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-18.us-west-1.compute.internal" event="NodeNotReady"
Oct 30 21:42:34.668362 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.668253    4473 setters.go:546] "Node became not ready" node="ip-10-0-140-18.us-west-1.compute.internal" condition={"type":"Ready","status":"False","lastHeartbeatTime":"2026-10-30T21:42:34Z","lastTransitionTime":"2026-10-30T21:42:34Z","reason":"KubeletNotReady","message":"CSINode is not yet initialized"}
Oct 30 21:42:34.677758 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.677703    4473 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 21:42:34.677863 ip-10-0-140-18 kubenswrapper[4473]: E1030 21:42:34.677757    4473 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-140-18.us-west-1.compute.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-18.us-west-1.compute.internal"
Oct 30 21:42:34.683020 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.682997    4473 nodeinfomanager.go:471] existing CSINode "ip-10-0-140-18.us-west-1.compute.internal" is owned by different node (oldNodeID="575cd0d8-2e45-4a75-b745-91afe4c781bd", newNodeID=""), cleaning up...
Oct 30 21:42:34.719525 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.719495    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/aae6ff4840f4ffd5842f280d0afa74f6-config\") pod \"kube-apiserver-proxy-ip-10-0-140-18.us-west-1.compute.internal\" (UID: \"aae6ff4840f4ffd5842f280d0afa74f6\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-18.us-west-1.compute.internal"
Oct 30 21:42:34.719686 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.719532    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/9a45018bbca93ea1080adf030d8fbb7f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-18.us-west-1.compute.internal\" (UID: \"9a45018bbca93ea1080adf030d8fbb7f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-18.us-west-1.compute.internal"
Oct 30 21:42:34.719686 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.719550    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9a45018bbca93ea1080adf030d8fbb7f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-18.us-west-1.compute.internal\" (UID: \"9a45018bbca93ea1080adf030d8fbb7f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-18.us-west-1.compute.internal"
Oct 30 21:42:34.719686 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.719571    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/aae6ff4840f4ffd5842f280d0afa74f6-config\") pod \"kube-apiserver-proxy-ip-10-0-140-18.us-west-1.compute.internal\" (UID: \"aae6ff4840f4ffd5842f280d0afa74f6\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-18.us-west-1.compute.internal"
Oct 30 21:42:34.719686 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.719595    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9a45018bbca93ea1080adf030d8fbb7f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-18.us-west-1.compute.internal\" (UID: \"9a45018bbca93ea1080adf030d8fbb7f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-18.us-west-1.compute.internal"
Oct 30 21:42:34.719686 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:34.719610    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/9a45018bbca93ea1080adf030d8fbb7f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-18.us-west-1.compute.internal\" (UID: \"9a45018bbca93ea1080adf030d8fbb7f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-18.us-west-1.compute.internal"
Oct 30 21:42:35.305621 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.305586    4473 apiserver.go:52] "Watching apiserver"
Oct 30 21:42:35.391668 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.391639    4473 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-18.us-west-1.compute.internal"
Oct 30 21:42:35.391826 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.391736    4473 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-18.us-west-1.compute.internal"
Oct 30 21:42:35.410693 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.410665    4473 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Oct 30 21:42:35.411147 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.411129    4473 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-hbmxt","openshift-kube-storage-version-migrator/migrator-66df5d496b-gj8rz","openshift-ovn-kubernetes/ovnkube-node-w6d9j","kube-system/konnectivity-agent-krcxf","openshift-multus/multus-additional-cni-plugins-kkr27","kube-system/kube-apiserver-proxy-ip-10-0-140-18.us-west-1.compute.internal","openshift-insights/periodic-gathering-bcmkf-fm7wg","openshift-network-operator/iptables-alerter-bgfld","openshift-network-diagnostics/network-check-target-9st74","openshift-image-registry/node-ca-jwvtb","kube-system/global-pull-secret-syncer-dt7cq","openshift-multus/multus-hs4mm","openshift-multus/network-metrics-daemon-8jvqh","openshift-dns/node-resolver-46nkk","openshift-cluster-node-tuning-operator/tuned-92gv2","openshift-insights/insights-runtime-extractor-fncvg","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8lm5l","openshift-ingress-canary/ingress-canary-5m6cr","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-18.us-west-1.compute.internal"]
Oct 30 21:42:35.411591 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.411573    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8jvqh"
Oct 30 21:42:35.415683 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.415664    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-hbmxt"
Oct 30 21:42:35.416505 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.416491    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-fncvg"
Oct 30 21:42:35.420298 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.420273    4473 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Oct 30 21:42:35.423316 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.423295    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c67fe06a-8f45-48d0-9451-b9157f57369e-etc-systemd\") pod \"tuned-92gv2\" (UID: \"c67fe06a-8f45-48d0-9451-b9157f57369e\") " pod="openshift-cluster-node-tuning-operator/tuned-92gv2"
Oct 30 21:42:35.423385 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.423320    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/03a62eeb-c318-4ce3-b6e9-1afba98920e9-host-run-netns\") pod \"multus-hs4mm\" (UID: \"03a62eeb-c318-4ce3-b6e9-1afba98920e9\") " pod="openshift-multus/multus-hs4mm"
Oct 30 21:42:35.423385 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.423335    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/2634eb5b-4ce8-4742-819a-616dcd33da39-hosts-file\") pod \"node-resolver-46nkk\" (UID: \"2634eb5b-4ce8-4742-819a-616dcd33da39\") " pod="openshift-dns/node-resolver-46nkk"
Oct 30 21:42:35.423385 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.423353    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/bddede0e-ee07-4344-878d-10b967435e13-archives-path\") pod \"periodic-gathering-bcmkf-fm7wg\" (UID: \"bddede0e-ee07-4344-878d-10b967435e13\") " pod="openshift-insights/periodic-gathering-bcmkf-fm7wg"
Oct 30 21:42:35.423385 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.423368    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/5de0b6c7-0d1c-4732-abce-361a46df75c3-etc-selinux\") pod \"aws-ebs-csi-driver-node-8lm5l\" (UID: \"5de0b6c7-0d1c-4732-abce-361a46df75c3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8lm5l"
Oct 30 21:42:35.423579 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.423451    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/719f98aa-6247-4973-8d2c-a91aaf276984-host-slash\") pod \"iptables-alerter-bgfld\" (UID: \"719f98aa-6247-4973-8d2c-a91aaf276984\") " pod="openshift-network-operator/iptables-alerter-bgfld"
Oct 30 21:42:35.423579 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.423504    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-js6zb\" (UniqueName: \"kubernetes.io/projected/10d2e5ae-c1c4-400c-8c67-3aebb6caaa6a-kube-api-access-js6zb\") pod \"network-check-target-9st74\" (UID: \"10d2e5ae-c1c4-400c-8c67-3aebb6caaa6a\") " pod="openshift-network-diagnostics/network-check-target-9st74"
Oct 30 21:42:35.423579 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.423531    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/bddede0e-ee07-4344-878d-10b967435e13-archives-path\") pod \"periodic-gathering-bcmkf-fm7wg\" (UID: \"bddede0e-ee07-4344-878d-10b967435e13\") " pod="openshift-insights/periodic-gathering-bcmkf-fm7wg"
Oct 30 21:42:35.423776 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.423577    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/641ac65e-00db-4646-b03c-9e3cb0c70feb-var-lib-openvswitch\") pod \"ovnkube-node-w6d9j\" (UID: \"641ac65e-00db-4646-b03c-9e3cb0c70feb\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6d9j"
Oct 30 21:42:35.423776 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.423707    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/641ac65e-00db-4646-b03c-9e3cb0c70feb-run-ovn\") pod \"ovnkube-node-w6d9j\" (UID: \"641ac65e-00db-4646-b03c-9e3cb0c70feb\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6d9j"
Oct 30 21:42:35.423776 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.423724    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/641ac65e-00db-4646-b03c-9e3cb0c70feb-ovnkube-script-lib\") pod \"ovnkube-node-w6d9j\" (UID: \"641ac65e-00db-4646-b03c-9e3cb0c70feb\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6d9j"
Oct 30 21:42:35.423776 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.423740    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f152360c-cf62-4d10-b2e5-b85ac7dfb739-metrics-tls\") pod \"dns-default-hbmxt\" (UID: \"f152360c-cf62-4d10-b2e5-b85ac7dfb739\") " pod="openshift-dns/dns-default-hbmxt"
Oct 30 21:42:35.423776 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.423754    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/5de0b6c7-0d1c-4732-abce-361a46df75c3-registration-dir\") pod \"aws-ebs-csi-driver-node-8lm5l\" (UID: \"5de0b6c7-0d1c-4732-abce-361a46df75c3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8lm5l"
Oct 30 21:42:35.423776 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.423770    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/641ac65e-00db-4646-b03c-9e3cb0c70feb-env-overrides\") pod \"ovnkube-node-w6d9j\" (UID: \"641ac65e-00db-4646-b03c-9e3cb0c70feb\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6d9j"
Oct 30 21:42:35.424001 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.423809    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/03a62eeb-c318-4ce3-b6e9-1afba98920e9-multus-cni-dir\") pod \"multus-hs4mm\" (UID: \"03a62eeb-c318-4ce3-b6e9-1afba98920e9\") " pod="openshift-multus/multus-hs4mm"
Oct 30 21:42:35.424001 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.423848    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/03a62eeb-c318-4ce3-b6e9-1afba98920e9-host-var-lib-cni-bin\") pod \"multus-hs4mm\" (UID: \"03a62eeb-c318-4ce3-b6e9-1afba98920e9\") " pod="openshift-multus/multus-hs4mm"
Oct 30 21:42:35.424001 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.423903    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/03a62eeb-c318-4ce3-b6e9-1afba98920e9-host-run-multus-certs\") pod \"multus-hs4mm\" (UID: \"03a62eeb-c318-4ce3-b6e9-1afba98920e9\") " pod="openshift-multus/multus-hs4mm"
Oct 30 21:42:35.424001 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.423951    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bddede0e-ee07-4344-878d-10b967435e13-service-ca-bundle\") pod \"periodic-gathering-bcmkf-fm7wg\" (UID: \"bddede0e-ee07-4344-878d-10b967435e13\") " pod="openshift-insights/periodic-gathering-bcmkf-fm7wg"
Oct 30 21:42:35.424180 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.424004    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/5de0b6c7-0d1c-4732-abce-361a46df75c3-kubelet-dir\") pod \"aws-ebs-csi-driver-node-8lm5l\" (UID: \"5de0b6c7-0d1c-4732-abce-361a46df75c3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8lm5l"
Oct 30 21:42:35.424180 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.424032    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/5de0b6c7-0d1c-4732-abce-361a46df75c3-socket-dir\") pod \"aws-ebs-csi-driver-node-8lm5l\" (UID: \"5de0b6c7-0d1c-4732-abce-361a46df75c3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8lm5l"
Oct 30 21:42:35.424180 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.424055    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/641ac65e-00db-4646-b03c-9e3cb0c70feb-host-slash\") pod \"ovnkube-node-w6d9j\" (UID: \"641ac65e-00db-4646-b03c-9e3cb0c70feb\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6d9j"
Oct 30 21:42:35.424180 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.424077    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/641ac65e-00db-4646-b03c-9e3cb0c70feb-etc-openvswitch\") pod \"ovnkube-node-w6d9j\" (UID: \"641ac65e-00db-4646-b03c-9e3cb0c70feb\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6d9j"
Oct 30 21:42:35.424180 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.424103    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/641ac65e-00db-4646-b03c-9e3cb0c70feb-host-cni-netd\") pod \"ovnkube-node-w6d9j\" (UID: \"641ac65e-00db-4646-b03c-9e3cb0c70feb\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6d9j"
Oct 30 21:42:35.424180 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.424117    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c67fe06a-8f45-48d0-9451-b9157f57369e-run\") pod \"tuned-92gv2\" (UID: \"c67fe06a-8f45-48d0-9451-b9157f57369e\") " pod="openshift-cluster-node-tuning-operator/tuned-92gv2"
Oct 30 21:42:35.424180 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.424178    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c67fe06a-8f45-48d0-9451-b9157f57369e-etc-tuned\") pod \"tuned-92gv2\" (UID: \"c67fe06a-8f45-48d0-9451-b9157f57369e\") " pod="openshift-cluster-node-tuning-operator/tuned-92gv2"
Oct 30 21:42:35.424492 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.424202    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lnn4t\" (UniqueName: \"kubernetes.io/projected/bddede0e-ee07-4344-878d-10b967435e13-kube-api-access-lnn4t\") pod \"periodic-gathering-bcmkf-fm7wg\" (UID: \"bddede0e-ee07-4344-878d-10b967435e13\") " pod="openshift-insights/periodic-gathering-bcmkf-fm7wg"
Oct 30 21:42:35.424492 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.424241    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0743c782-f5ad-4d3a-86e7-fc66563902c7-cnibin\") pod \"multus-additional-cni-plugins-kkr27\" (UID: \"0743c782-f5ad-4d3a-86e7-fc66563902c7\") " pod="openshift-multus/multus-additional-cni-plugins-kkr27"
Oct 30 21:42:35.424492 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.424275    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/0743c782-f5ad-4d3a-86e7-fc66563902c7-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-kkr27\" (UID: \"0743c782-f5ad-4d3a-86e7-fc66563902c7\") " pod="openshift-multus/multus-additional-cni-plugins-kkr27"
Oct 30 21:42:35.424492 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.424312    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/641ac65e-00db-4646-b03c-9e3cb0c70feb-host-kubelet\") pod \"ovnkube-node-w6d9j\" (UID: \"641ac65e-00db-4646-b03c-9e3cb0c70feb\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6d9j"
Oct 30 21:42:35.424492 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.424338    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c67fe06a-8f45-48d0-9451-b9157f57369e-etc-tuned\") pod \"tuned-92gv2\" (UID: \"c67fe06a-8f45-48d0-9451-b9157f57369e\") " pod="openshift-cluster-node-tuning-operator/tuned-92gv2"
Oct 30 21:42:35.424492 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.424361    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/03a62eeb-c318-4ce3-b6e9-1afba98920e9-host-run-k8s-cni-cncf-io\") pod \"multus-hs4mm\" (UID: \"03a62eeb-c318-4ce3-b6e9-1afba98920e9\") " pod="openshift-multus/multus-hs4mm"
Oct 30 21:42:35.424492 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.424384    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/03a62eeb-c318-4ce3-b6e9-1afba98920e9-host-var-lib-kubelet\") pod \"multus-hs4mm\" (UID: \"03a62eeb-c318-4ce3-b6e9-1afba98920e9\") " pod="openshift-multus/multus-hs4mm"
Oct 30 21:42:35.424492 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.424414    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n6qcb\" (UniqueName: \"kubernetes.io/projected/f152360c-cf62-4d10-b2e5-b85ac7dfb739-kube-api-access-n6qcb\") pod \"dns-default-hbmxt\" (UID: \"f152360c-cf62-4d10-b2e5-b85ac7dfb739\") " pod="openshift-dns/dns-default-hbmxt"
Oct 30 21:42:35.424492 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.424435    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0743c782-f5ad-4d3a-86e7-fc66563902c7-tuning-conf-dir\") pod \"multus-additional-cni-plugins-kkr27\" (UID: \"0743c782-f5ad-4d3a-86e7-fc66563902c7\") " pod="openshift-multus/multus-additional-cni-plugins-kkr27"
Oct 30 21:42:35.424492 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.424449    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/5de0b6c7-0d1c-4732-abce-361a46df75c3-device-dir\") pod \"aws-ebs-csi-driver-node-8lm5l\" (UID: \"5de0b6c7-0d1c-4732-abce-361a46df75c3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8lm5l"
Oct 30 21:42:35.424492 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.424470    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/641ac65e-00db-4646-b03c-9e3cb0c70feb-host-cni-bin\") pod \"ovnkube-node-w6d9j\" (UID: \"641ac65e-00db-4646-b03c-9e3cb0c70feb\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6d9j"
Oct 30 21:42:35.424492 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.424494    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c67fe06a-8f45-48d0-9451-b9157f57369e-etc-sysctl-conf\") pod \"tuned-92gv2\" (UID: \"c67fe06a-8f45-48d0-9451-b9157f57369e\") " pod="openshift-cluster-node-tuning-operator/tuned-92gv2"
Oct 30 21:42:35.425052 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.424535    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c67fe06a-8f45-48d0-9451-b9157f57369e-host\") pod \"tuned-92gv2\" (UID: \"c67fe06a-8f45-48d0-9451-b9157f57369e\") " pod="openshift-cluster-node-tuning-operator/tuned-92gv2"
Oct 30 21:42:35.425052 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.424551    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2428ae10-7385-4056-aabe-bdf1b974a36b-host\") pod \"node-ca-jwvtb\" (UID: \"2428ae10-7385-4056-aabe-bdf1b974a36b\") " pod="openshift-image-registry/node-ca-jwvtb"
Oct 30 21:42:35.425052 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.424566    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/06552eb8-7379-40df-b31c-a133e646afb9-agent-certs\") pod \"konnectivity-agent-krcxf\" (UID: \"06552eb8-7379-40df-b31c-a133e646afb9\") " pod="kube-system/konnectivity-agent-krcxf"
Oct 30 21:42:35.425052 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.424587    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/641ac65e-00db-4646-b03c-9e3cb0c70feb-host-run-netns\") pod \"ovnkube-node-w6d9j\" (UID: \"641ac65e-00db-4646-b03c-9e3cb0c70feb\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6d9j"
Oct 30 21:42:35.425052 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.424628    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bddede0e-ee07-4344-878d-10b967435e13-trusted-ca-bundle\") pod \"periodic-gathering-bcmkf-fm7wg\" (UID: \"bddede0e-ee07-4344-878d-10b967435e13\") " pod="openshift-insights/periodic-gathering-bcmkf-fm7wg"
Oct 30 21:42:35.425052 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.424694    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/0032cb99-3a37-40c6-98cb-2d40ae4a9319-dbus\") pod \"global-pull-secret-syncer-dt7cq\" (UID: \"0032cb99-3a37-40c6-98cb-2d40ae4a9319\") " pod="kube-system/global-pull-secret-syncer-dt7cq"
Oct 30 21:42:35.425052 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.424719    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/03a62eeb-c318-4ce3-b6e9-1afba98920e9-multus-socket-dir-parent\") pod \"multus-hs4mm\" (UID: \"03a62eeb-c318-4ce3-b6e9-1afba98920e9\") " pod="openshift-multus/multus-hs4mm"
Oct 30 21:42:35.425052 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.424749    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/03a62eeb-c318-4ce3-b6e9-1afba98920e9-hostroot\") pod \"multus-hs4mm\" (UID: \"03a62eeb-c318-4ce3-b6e9-1afba98920e9\") " pod="openshift-multus/multus-hs4mm"
Oct 30 21:42:35.425052 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.424777    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/2428ae10-7385-4056-aabe-bdf1b974a36b-serviceca\") pod \"node-ca-jwvtb\" (UID: \"2428ae10-7385-4056-aabe-bdf1b974a36b\") " pod="openshift-image-registry/node-ca-jwvtb"
Oct 30 21:42:35.425052 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.424808    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/06552eb8-7379-40df-b31c-a133e646afb9-konnectivity-ca\") pod \"konnectivity-agent-krcxf\" (UID: \"06552eb8-7379-40df-b31c-a133e646afb9\") " pod="kube-system/konnectivity-agent-krcxf"
Oct 30 21:42:35.425052 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.424868    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0743c782-f5ad-4d3a-86e7-fc66563902c7-system-cni-dir\") pod \"multus-additional-cni-plugins-kkr27\" (UID: \"0743c782-f5ad-4d3a-86e7-fc66563902c7\") " pod="openshift-multus/multus-additional-cni-plugins-kkr27"
Oct 30 21:42:35.425052 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.424885    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0743c782-f5ad-4d3a-86e7-fc66563902c7-cni-binary-copy\") pod \"multus-additional-cni-plugins-kkr27\" (UID: \"0743c782-f5ad-4d3a-86e7-fc66563902c7\") " pod="openshift-multus/multus-additional-cni-plugins-kkr27"
Oct 30 21:42:35.425052 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.424901    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7vks4\" (UniqueName: \"kubernetes.io/projected/4436ec50-e190-4c1c-8b6c-2b9b5d12cb9f-kube-api-access-7vks4\") pod \"ingress-canary-5m6cr\" (UID: \"4436ec50-e190-4c1c-8b6c-2b9b5d12cb9f\") " pod="openshift-ingress-canary/ingress-canary-5m6cr"
Oct 30 21:42:35.425052 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.424934    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/32a1c660-effa-4b6c-975a-dbb68c468eda-data-volume\") pod \"insights-runtime-extractor-fncvg\" (UID: \"32a1c660-effa-4b6c-975a-dbb68c468eda\") " pod="openshift-insights/insights-runtime-extractor-fncvg"
Oct 30 21:42:35.425052 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.425002    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/32a1c660-effa-4b6c-975a-dbb68c468eda-data-volume\") pod \"insights-runtime-extractor-fncvg\" (UID: \"32a1c660-effa-4b6c-975a-dbb68c468eda\") " pod="openshift-insights/insights-runtime-extractor-fncvg"
Oct 30 21:42:35.425052 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.425044    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/641ac65e-00db-4646-b03c-9e3cb0c70feb-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-w6d9j\" (UID: \"641ac65e-00db-4646-b03c-9e3cb0c70feb\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6d9j"
Oct 30 21:42:35.425052 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.425061    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c67fe06a-8f45-48d0-9451-b9157f57369e-etc-sysconfig\") pod \"tuned-92gv2\" (UID: \"c67fe06a-8f45-48d0-9451-b9157f57369e\") " pod="openshift-cluster-node-tuning-operator/tuned-92gv2"
Oct 30 21:42:35.425575 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.425076    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c67fe06a-8f45-48d0-9451-b9157f57369e-var-lib-kubelet\") pod \"tuned-92gv2\" (UID: \"c67fe06a-8f45-48d0-9451-b9157f57369e\") " pod="openshift-cluster-node-tuning-operator/tuned-92gv2"
Oct 30 21:42:35.425575 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.425098    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/32a1c660-effa-4b6c-975a-dbb68c468eda-crio-socket\") pod \"insights-runtime-extractor-fncvg\" (UID: \"32a1c660-effa-4b6c-975a-dbb68c468eda\") " pod="openshift-insights/insights-runtime-extractor-fncvg"
Oct 30 21:42:35.425575 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.425125    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/32a1c660-effa-4b6c-975a-dbb68c468eda-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fncvg\" (UID: \"32a1c660-effa-4b6c-975a-dbb68c468eda\") " pod="openshift-insights/insights-runtime-extractor-fncvg"
Oct 30 21:42:35.425575 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.425160    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/641ac65e-00db-4646-b03c-9e3cb0c70feb-run-systemd\") pod \"ovnkube-node-w6d9j\" (UID: \"641ac65e-00db-4646-b03c-9e3cb0c70feb\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6d9j"
Oct 30 21:42:35.425575 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.425197    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/641ac65e-00db-4646-b03c-9e3cb0c70feb-node-log\") pod \"ovnkube-node-w6d9j\" (UID: \"641ac65e-00db-4646-b03c-9e3cb0c70feb\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6d9j"
Oct 30 21:42:35.425575 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.425232    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c67fe06a-8f45-48d0-9451-b9157f57369e-lib-modules\") pod \"tuned-92gv2\" (UID: \"c67fe06a-8f45-48d0-9451-b9157f57369e\") " pod="openshift-cluster-node-tuning-operator/tuned-92gv2"
Oct 30 21:42:35.425575 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.425248    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/03a62eeb-c318-4ce3-b6e9-1afba98920e9-system-cni-dir\") pod \"multus-hs4mm\" (UID: \"03a62eeb-c318-4ce3-b6e9-1afba98920e9\") " pod="openshift-multus/multus-hs4mm"
Oct 30 21:42:35.425575 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.425266    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g98k9\" (UniqueName: \"kubernetes.io/projected/525a0437-9a44-425b-966a-b118d91a4522-kube-api-access-g98k9\") pod \"migrator-66df5d496b-gj8rz\" (UID: \"525a0437-9a44-425b-966a-b118d91a4522\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-gj8rz"
Oct 30 21:42:35.425575 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.425287    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f152360c-cf62-4d10-b2e5-b85ac7dfb739-tmp-dir\") pod \"dns-default-hbmxt\" (UID: \"f152360c-cf62-4d10-b2e5-b85ac7dfb739\") " pod="openshift-dns/dns-default-hbmxt"
Oct 30 21:42:35.425575 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.425316    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4436ec50-e190-4c1c-8b6c-2b9b5d12cb9f-cert\") pod \"ingress-canary-5m6cr\" (UID: \"4436ec50-e190-4c1c-8b6c-2b9b5d12cb9f\") " pod="openshift-ingress-canary/ingress-canary-5m6cr"
Oct 30 21:42:35.425575 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.425344    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f152360c-cf62-4d10-b2e5-b85ac7dfb739-tmp-dir\") pod \"dns-default-hbmxt\" (UID: \"f152360c-cf62-4d10-b2e5-b85ac7dfb739\") " pod="openshift-dns/dns-default-hbmxt"
Oct 30 21:42:35.425575 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.425393    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/641ac65e-00db-4646-b03c-9e3cb0c70feb-systemd-units\") pod \"ovnkube-node-w6d9j\" (UID: \"641ac65e-00db-4646-b03c-9e3cb0c70feb\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6d9j"
Oct 30 21:42:35.425575 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.425456    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c67fe06a-8f45-48d0-9451-b9157f57369e-etc-sysctl-d\") pod \"tuned-92gv2\" (UID: \"c67fe06a-8f45-48d0-9451-b9157f57369e\") " pod="openshift-cluster-node-tuning-operator/tuned-92gv2"
Oct 30 21:42:35.425575 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.425485    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/03a62eeb-c318-4ce3-b6e9-1afba98920e9-multus-conf-dir\") pod \"multus-hs4mm\" (UID: \"03a62eeb-c318-4ce3-b6e9-1afba98920e9\") " pod="openshift-multus/multus-hs4mm"
Oct 30 21:42:35.425575 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.425506    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/f152360c-cf62-4d10-b2e5-b85ac7dfb739-config-volume\") pod \"dns-default-hbmxt\" (UID: \"f152360c-cf62-4d10-b2e5-b85ac7dfb739\") " pod="openshift-dns/dns-default-hbmxt"
Oct 30 21:42:35.425575 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.425541    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/641ac65e-00db-4646-b03c-9e3cb0c70feb-ovnkube-config\") pod \"ovnkube-node-w6d9j\" (UID: \"641ac65e-00db-4646-b03c-9e3cb0c70feb\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6d9j"
Oct 30 21:42:35.426077 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.425600    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/641ac65e-00db-4646-b03c-9e3cb0c70feb-ovn-node-metrics-cert\") pod \"ovnkube-node-w6d9j\" (UID: \"641ac65e-00db-4646-b03c-9e3cb0c70feb\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6d9j"
Oct 30 21:42:35.426077 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.425631    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/03a62eeb-c318-4ce3-b6e9-1afba98920e9-cnibin\") pod \"multus-hs4mm\" (UID: \"03a62eeb-c318-4ce3-b6e9-1afba98920e9\") " pod="openshift-multus/multus-hs4mm"
Oct 30 21:42:35.426077 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.425679    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/03a62eeb-c318-4ce3-b6e9-1afba98920e9-cni-binary-copy\") pod \"multus-hs4mm\" (UID: \"03a62eeb-c318-4ce3-b6e9-1afba98920e9\") " pod="openshift-multus/multus-hs4mm"
Oct 30 21:42:35.426077 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.425697    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2634eb5b-4ce8-4742-819a-616dcd33da39-tmp-dir\") pod \"node-resolver-46nkk\" (UID: \"2634eb5b-4ce8-4742-819a-616dcd33da39\") " pod="openshift-dns/node-resolver-46nkk"
Oct 30 21:42:35.426077 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.425714    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-htnvw\" (UniqueName: \"kubernetes.io/projected/2634eb5b-4ce8-4742-819a-616dcd33da39-kube-api-access-htnvw\") pod \"node-resolver-46nkk\" (UID: \"2634eb5b-4ce8-4742-819a-616dcd33da39\") " pod="openshift-dns/node-resolver-46nkk"
Oct 30 21:42:35.426077 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.425735    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0032cb99-3a37-40c6-98cb-2d40ae4a9319-original-pull-secret\") pod \"global-pull-secret-syncer-dt7cq\" (UID: \"0032cb99-3a37-40c6-98cb-2d40ae4a9319\") " pod="kube-system/global-pull-secret-syncer-dt7cq"
Oct 30 21:42:35.426077 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.425758    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pn9mk\" (UniqueName: \"kubernetes.io/projected/719f98aa-6247-4973-8d2c-a91aaf276984-kube-api-access-pn9mk\") pod \"iptables-alerter-bgfld\" (UID: \"719f98aa-6247-4973-8d2c-a91aaf276984\") " pod="openshift-network-operator/iptables-alerter-bgfld"
Oct 30 21:42:35.426077 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.425787    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2634eb5b-4ce8-4742-819a-616dcd33da39-tmp-dir\") pod \"node-resolver-46nkk\" (UID: \"2634eb5b-4ce8-4742-819a-616dcd33da39\") " pod="openshift-dns/node-resolver-46nkk"
Oct 30 21:42:35.426077 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.425821    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/641ac65e-00db-4646-b03c-9e3cb0c70feb-host-run-ovn-kubernetes\") pod \"ovnkube-node-w6d9j\" (UID: \"641ac65e-00db-4646-b03c-9e3cb0c70feb\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6d9j"
Oct 30 21:42:35.426077 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.425839    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c67fe06a-8f45-48d0-9451-b9157f57369e-etc-modprobe-d\") pod \"tuned-92gv2\" (UID: \"c67fe06a-8f45-48d0-9451-b9157f57369e\") " pod="openshift-cluster-node-tuning-operator/tuned-92gv2"
Oct 30 21:42:35.426077 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.425874    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c67fe06a-8f45-48d0-9451-b9157f57369e-sys\") pod \"tuned-92gv2\" (UID: \"c67fe06a-8f45-48d0-9451-b9157f57369e\") " pod="openshift-cluster-node-tuning-operator/tuned-92gv2"
Oct 30 21:42:35.426077 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.425894    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c67fe06a-8f45-48d0-9451-b9157f57369e-tmp\") pod \"tuned-92gv2\" (UID: \"c67fe06a-8f45-48d0-9451-b9157f57369e\") " pod="openshift-cluster-node-tuning-operator/tuned-92gv2"
Oct 30 21:42:35.426077 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.425932    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b8x4j\" (UniqueName: \"kubernetes.io/projected/ccc5af13-906a-469d-9ca5-09366d1a7e93-kube-api-access-b8x4j\") pod \"network-metrics-daemon-8jvqh\" (UID: \"ccc5af13-906a-469d-9ca5-09366d1a7e93\") " pod="openshift-multus/network-metrics-daemon-8jvqh"
Oct 30 21:42:35.426077 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.425957    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r9bdf\" (UniqueName: \"kubernetes.io/projected/0743c782-f5ad-4d3a-86e7-fc66563902c7-kube-api-access-r9bdf\") pod \"multus-additional-cni-plugins-kkr27\" (UID: \"0743c782-f5ad-4d3a-86e7-fc66563902c7\") " pod="openshift-multus/multus-additional-cni-plugins-kkr27"
Oct 30 21:42:35.426077 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.426019    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/32a1c660-effa-4b6c-975a-dbb68c468eda-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fncvg\" (UID: \"32a1c660-effa-4b6c-975a-dbb68c468eda\") " pod="openshift-insights/insights-runtime-extractor-fncvg"
Oct 30 21:42:35.426077 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.426042    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c67fe06a-8f45-48d0-9451-b9157f57369e-tmp\") pod \"tuned-92gv2\" (UID: \"c67fe06a-8f45-48d0-9451-b9157f57369e\") " pod="openshift-cluster-node-tuning-operator/tuned-92gv2"
Oct 30 21:42:35.426077 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.426069    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zzqrt\" (UniqueName: \"kubernetes.io/projected/32a1c660-effa-4b6c-975a-dbb68c468eda-kube-api-access-zzqrt\") pod \"insights-runtime-extractor-fncvg\" (UID: \"32a1c660-effa-4b6c-975a-dbb68c468eda\") " pod="openshift-insights/insights-runtime-extractor-fncvg"
Oct 30 21:42:35.426746 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.426096    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-64s8x\" (UniqueName: \"kubernetes.io/projected/c67fe06a-8f45-48d0-9451-b9157f57369e-kube-api-access-64s8x\") pod \"tuned-92gv2\" (UID: \"c67fe06a-8f45-48d0-9451-b9157f57369e\") " pod="openshift-cluster-node-tuning-operator/tuned-92gv2"
Oct 30 21:42:35.426746 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.426158    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/03a62eeb-c318-4ce3-b6e9-1afba98920e9-host-var-lib-cni-multus\") pod \"multus-hs4mm\" (UID: \"03a62eeb-c318-4ce3-b6e9-1afba98920e9\") " pod="openshift-multus/multus-hs4mm"
Oct 30 21:42:35.426746 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.426181    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5xlxd\" (UniqueName: \"kubernetes.io/projected/2428ae10-7385-4056-aabe-bdf1b974a36b-kube-api-access-5xlxd\") pod \"node-ca-jwvtb\" (UID: \"2428ae10-7385-4056-aabe-bdf1b974a36b\") " pod="openshift-image-registry/node-ca-jwvtb"
Oct 30 21:42:35.426746 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.426243    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/0032cb99-3a37-40c6-98cb-2d40ae4a9319-kubelet-config\") pod \"global-pull-secret-syncer-dt7cq\" (UID: \"0032cb99-3a37-40c6-98cb-2d40ae4a9319\") " pod="kube-system/global-pull-secret-syncer-dt7cq"
Oct 30 21:42:35.426746 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.426264    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/719f98aa-6247-4973-8d2c-a91aaf276984-iptables-alerter-script\") pod \"iptables-alerter-bgfld\" (UID: \"719f98aa-6247-4973-8d2c-a91aaf276984\") " pod="openshift-network-operator/iptables-alerter-bgfld"
Oct 30 21:42:35.426746 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.426279    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/641ac65e-00db-4646-b03c-9e3cb0c70feb-run-openvswitch\") pod \"ovnkube-node-w6d9j\" (UID: \"641ac65e-00db-4646-b03c-9e3cb0c70feb\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6d9j"
Oct 30 21:42:35.426746 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.426294    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9bq8j\" (UniqueName: \"kubernetes.io/projected/03a62eeb-c318-4ce3-b6e9-1afba98920e9-kube-api-access-9bq8j\") pod \"multus-hs4mm\" (UID: \"03a62eeb-c318-4ce3-b6e9-1afba98920e9\") " pod="openshift-multus/multus-hs4mm"
Oct 30 21:42:35.426746 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.426336    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ccc5af13-906a-469d-9ca5-09366d1a7e93-metrics-certs\") pod \"network-metrics-daemon-8jvqh\" (UID: \"ccc5af13-906a-469d-9ca5-09366d1a7e93\") " pod="openshift-multus/network-metrics-daemon-8jvqh"
Oct 30 21:42:35.426746 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.426373    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/0743c782-f5ad-4d3a-86e7-fc66563902c7-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-kkr27\" (UID: \"0743c782-f5ad-4d3a-86e7-fc66563902c7\") " pod="openshift-multus/multus-additional-cni-plugins-kkr27"
Oct 30 21:42:35.426746 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.426449    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/641ac65e-00db-4646-b03c-9e3cb0c70feb-log-socket\") pod \"ovnkube-node-w6d9j\" (UID: \"641ac65e-00db-4646-b03c-9e3cb0c70feb\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6d9j"
Oct 30 21:42:35.426746 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.426483    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9hm6j\" (UniqueName: \"kubernetes.io/projected/641ac65e-00db-4646-b03c-9e3cb0c70feb-kube-api-access-9hm6j\") pod \"ovnkube-node-w6d9j\" (UID: \"641ac65e-00db-4646-b03c-9e3cb0c70feb\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6d9j"
Oct 30 21:42:35.426746 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.426508    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/03a62eeb-c318-4ce3-b6e9-1afba98920e9-multus-daemon-config\") pod \"multus-hs4mm\" (UID: \"03a62eeb-c318-4ce3-b6e9-1afba98920e9\") " pod="openshift-multus/multus-hs4mm"
Oct 30 21:42:35.426746 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.426535    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/03a62eeb-c318-4ce3-b6e9-1afba98920e9-etc-kubernetes\") pod \"multus-hs4mm\" (UID: \"03a62eeb-c318-4ce3-b6e9-1afba98920e9\") " pod="openshift-multus/multus-hs4mm"
Oct 30 21:42:35.426746 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.426560    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/5de0b6c7-0d1c-4732-abce-361a46df75c3-sys-fs\") pod \"aws-ebs-csi-driver-node-8lm5l\" (UID: \"5de0b6c7-0d1c-4732-abce-361a46df75c3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8lm5l"
Oct 30 21:42:35.426746 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.426592    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rfxjl\" (UniqueName: \"kubernetes.io/projected/5de0b6c7-0d1c-4732-abce-361a46df75c3-kube-api-access-rfxjl\") pod \"aws-ebs-csi-driver-node-8lm5l\" (UID: \"5de0b6c7-0d1c-4732-abce-361a46df75c3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8lm5l"
Oct 30 21:42:35.426746 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.426620    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c67fe06a-8f45-48d0-9451-b9157f57369e-etc-kubernetes\") pod \"tuned-92gv2\" (UID: \"c67fe06a-8f45-48d0-9451-b9157f57369e\") " pod="openshift-cluster-node-tuning-operator/tuned-92gv2"
Oct 30 21:42:35.452662 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.452643    4473 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 21:42:35.452747 ip-10-0-140-18 kubenswrapper[4473]: E1030 21:42:35.452688    4473 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-140-18.us-west-1.compute.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-18.us-west-1.compute.internal"
Oct 30 21:42:35.453175 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.453162    4473 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 21:42:35.453210 ip-10-0-140-18 kubenswrapper[4473]: E1030 21:42:35.453199    4473 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-140-18.us-west-1.compute.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-18.us-west-1.compute.internal"
Oct 30 21:42:35.465115 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.465075    4473 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Oct 30 21:42:35.465115 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.465083    4473 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Oct 30 21:42:35.465115 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.465103    4473 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Oct 30 21:42:35.465329 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.465161    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-k4c9l\""
Oct 30 21:42:35.465329 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.465168    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Oct 30 21:42:35.465329 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.465179    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-spllf\""
Oct 30 21:42:35.465741 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.465723    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Oct 30 21:42:35.475527 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.475505    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/06552eb8-7379-40df-b31c-a133e646afb9-konnectivity-ca\") pod \"konnectivity-agent-krcxf\" (UID: \"06552eb8-7379-40df-b31c-a133e646afb9\") " pod="kube-system/konnectivity-agent-krcxf"
Oct 30 21:42:35.475611 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.475546    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/06552eb8-7379-40df-b31c-a133e646afb9-agent-certs\") pod \"konnectivity-agent-krcxf\" (UID: \"06552eb8-7379-40df-b31c-a133e646afb9\") " pod="kube-system/konnectivity-agent-krcxf"
Oct 30 21:42:35.500620 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.500589    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9bq8j\" (UniqueName: \"kubernetes.io/projected/03a62eeb-c318-4ce3-b6e9-1afba98920e9-kube-api-access-9bq8j\") pod \"multus-hs4mm\" (UID: \"03a62eeb-c318-4ce3-b6e9-1afba98920e9\") " pod="openshift-multus/multus-hs4mm"
Oct 30 21:42:35.500726 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.500637    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b8x4j\" (UniqueName: \"kubernetes.io/projected/ccc5af13-906a-469d-9ca5-09366d1a7e93-kube-api-access-b8x4j\") pod \"network-metrics-daemon-8jvqh\" (UID: \"ccc5af13-906a-469d-9ca5-09366d1a7e93\") " pod="openshift-multus/network-metrics-daemon-8jvqh"
Oct 30 21:42:35.501036 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.501022    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r9bdf\" (UniqueName: \"kubernetes.io/projected/0743c782-f5ad-4d3a-86e7-fc66563902c7-kube-api-access-r9bdf\") pod \"multus-additional-cni-plugins-kkr27\" (UID: \"0743c782-f5ad-4d3a-86e7-fc66563902c7\") " pod="openshift-multus/multus-additional-cni-plugins-kkr27"
Oct 30 21:42:35.527502 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.527476    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/03a62eeb-c318-4ce3-b6e9-1afba98920e9-cnibin\") pod \"multus-hs4mm\" (UID: \"03a62eeb-c318-4ce3-b6e9-1afba98920e9\") " pod="openshift-multus/multus-hs4mm"
Oct 30 21:42:35.527642 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.527507    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/641ac65e-00db-4646-b03c-9e3cb0c70feb-host-run-ovn-kubernetes\") pod \"ovnkube-node-w6d9j\" (UID: \"641ac65e-00db-4646-b03c-9e3cb0c70feb\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6d9j"
Oct 30 21:42:35.527642 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.527525    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c67fe06a-8f45-48d0-9451-b9157f57369e-etc-modprobe-d\") pod \"tuned-92gv2\" (UID: \"c67fe06a-8f45-48d0-9451-b9157f57369e\") " pod="openshift-cluster-node-tuning-operator/tuned-92gv2"
Oct 30 21:42:35.527642 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.527539    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c67fe06a-8f45-48d0-9451-b9157f57369e-sys\") pod \"tuned-92gv2\" (UID: \"c67fe06a-8f45-48d0-9451-b9157f57369e\") " pod="openshift-cluster-node-tuning-operator/tuned-92gv2"
Oct 30 21:42:35.527642 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.527595    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c67fe06a-8f45-48d0-9451-b9157f57369e-sys\") pod \"tuned-92gv2\" (UID: \"c67fe06a-8f45-48d0-9451-b9157f57369e\") " pod="openshift-cluster-node-tuning-operator/tuned-92gv2"
Oct 30 21:42:35.527642 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.527594    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/03a62eeb-c318-4ce3-b6e9-1afba98920e9-cnibin\") pod \"multus-hs4mm\" (UID: \"03a62eeb-c318-4ce3-b6e9-1afba98920e9\") " pod="openshift-multus/multus-hs4mm"
Oct 30 21:42:35.527642 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.527595    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/641ac65e-00db-4646-b03c-9e3cb0c70feb-host-run-ovn-kubernetes\") pod \"ovnkube-node-w6d9j\" (UID: \"641ac65e-00db-4646-b03c-9e3cb0c70feb\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6d9j"
Oct 30 21:42:35.527642 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.527619    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/32a1c660-effa-4b6c-975a-dbb68c468eda-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fncvg\" (UID: \"32a1c660-effa-4b6c-975a-dbb68c468eda\") " pod="openshift-insights/insights-runtime-extractor-fncvg"
Oct 30 21:42:35.528008 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.527662    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/03a62eeb-c318-4ce3-b6e9-1afba98920e9-host-var-lib-cni-multus\") pod \"multus-hs4mm\" (UID: \"03a62eeb-c318-4ce3-b6e9-1afba98920e9\") " pod="openshift-multus/multus-hs4mm"
Oct 30 21:42:35.528008 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.527676    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c67fe06a-8f45-48d0-9451-b9157f57369e-etc-modprobe-d\") pod \"tuned-92gv2\" (UID: \"c67fe06a-8f45-48d0-9451-b9157f57369e\") " pod="openshift-cluster-node-tuning-operator/tuned-92gv2"
Oct 30 21:42:35.528008 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.527692    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/0032cb99-3a37-40c6-98cb-2d40ae4a9319-kubelet-config\") pod \"global-pull-secret-syncer-dt7cq\" (UID: \"0032cb99-3a37-40c6-98cb-2d40ae4a9319\") " pod="kube-system/global-pull-secret-syncer-dt7cq"
Oct 30 21:42:35.528008 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.527718    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/03a62eeb-c318-4ce3-b6e9-1afba98920e9-host-var-lib-cni-multus\") pod \"multus-hs4mm\" (UID: \"03a62eeb-c318-4ce3-b6e9-1afba98920e9\") " pod="openshift-multus/multus-hs4mm"
Oct 30 21:42:35.528008 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.527718    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/641ac65e-00db-4646-b03c-9e3cb0c70feb-run-openvswitch\") pod \"ovnkube-node-w6d9j\" (UID: \"641ac65e-00db-4646-b03c-9e3cb0c70feb\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6d9j"
Oct 30 21:42:35.528008 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.527749    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ccc5af13-906a-469d-9ca5-09366d1a7e93-metrics-certs\") pod \"network-metrics-daemon-8jvqh\" (UID: \"ccc5af13-906a-469d-9ca5-09366d1a7e93\") " pod="openshift-multus/network-metrics-daemon-8jvqh"
Oct 30 21:42:35.528008 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.527753    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/641ac65e-00db-4646-b03c-9e3cb0c70feb-run-openvswitch\") pod \"ovnkube-node-w6d9j\" (UID: \"641ac65e-00db-4646-b03c-9e3cb0c70feb\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6d9j"
Oct 30 21:42:35.528008 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.527759    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/0032cb99-3a37-40c6-98cb-2d40ae4a9319-kubelet-config\") pod \"global-pull-secret-syncer-dt7cq\" (UID: \"0032cb99-3a37-40c6-98cb-2d40ae4a9319\") " pod="kube-system/global-pull-secret-syncer-dt7cq"
Oct 30 21:42:35.528008 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.527775    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/641ac65e-00db-4646-b03c-9e3cb0c70feb-log-socket\") pod \"ovnkube-node-w6d9j\" (UID: \"641ac65e-00db-4646-b03c-9e3cb0c70feb\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6d9j"
Oct 30 21:42:35.528008 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.527792    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/03a62eeb-c318-4ce3-b6e9-1afba98920e9-etc-kubernetes\") pod \"multus-hs4mm\" (UID: \"03a62eeb-c318-4ce3-b6e9-1afba98920e9\") " pod="openshift-multus/multus-hs4mm"
Oct 30 21:42:35.528008 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.527809    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/5de0b6c7-0d1c-4732-abce-361a46df75c3-sys-fs\") pod \"aws-ebs-csi-driver-node-8lm5l\" (UID: \"5de0b6c7-0d1c-4732-abce-361a46df75c3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8lm5l"
Oct 30 21:42:35.528008 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.527828    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c67fe06a-8f45-48d0-9451-b9157f57369e-etc-kubernetes\") pod \"tuned-92gv2\" (UID: \"c67fe06a-8f45-48d0-9451-b9157f57369e\") " pod="openshift-cluster-node-tuning-operator/tuned-92gv2"
Oct 30 21:42:35.528008 ip-10-0-140-18 kubenswrapper[4473]: E1030 21:42:35.527848    4473 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Oct 30 21:42:35.528008 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.527855    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c67fe06a-8f45-48d0-9451-b9157f57369e-etc-systemd\") pod \"tuned-92gv2\" (UID: \"c67fe06a-8f45-48d0-9451-b9157f57369e\") " pod="openshift-cluster-node-tuning-operator/tuned-92gv2"
Oct 30 21:42:35.528008 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.527857    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/03a62eeb-c318-4ce3-b6e9-1afba98920e9-etc-kubernetes\") pod \"multus-hs4mm\" (UID: \"03a62eeb-c318-4ce3-b6e9-1afba98920e9\") " pod="openshift-multus/multus-hs4mm"
Oct 30 21:42:35.528008 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.527879    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/03a62eeb-c318-4ce3-b6e9-1afba98920e9-host-run-netns\") pod \"multus-hs4mm\" (UID: \"03a62eeb-c318-4ce3-b6e9-1afba98920e9\") " pod="openshift-multus/multus-hs4mm"
Oct 30 21:42:35.528008 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.527883    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/641ac65e-00db-4646-b03c-9e3cb0c70feb-log-socket\") pod \"ovnkube-node-w6d9j\" (UID: \"641ac65e-00db-4646-b03c-9e3cb0c70feb\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6d9j"
Oct 30 21:42:35.528008 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.527878    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/5de0b6c7-0d1c-4732-abce-361a46df75c3-sys-fs\") pod \"aws-ebs-csi-driver-node-8lm5l\" (UID: \"5de0b6c7-0d1c-4732-abce-361a46df75c3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8lm5l"
Oct 30 21:42:35.528008 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.527906    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/2634eb5b-4ce8-4742-819a-616dcd33da39-hosts-file\") pod \"node-resolver-46nkk\" (UID: \"2634eb5b-4ce8-4742-819a-616dcd33da39\") " pod="openshift-dns/node-resolver-46nkk"
Oct 30 21:42:35.528008 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.527914    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/03a62eeb-c318-4ce3-b6e9-1afba98920e9-host-run-netns\") pod \"multus-hs4mm\" (UID: \"03a62eeb-c318-4ce3-b6e9-1afba98920e9\") " pod="openshift-multus/multus-hs4mm"
Oct 30 21:42:35.528008 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.527931    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c67fe06a-8f45-48d0-9451-b9157f57369e-etc-systemd\") pod \"tuned-92gv2\" (UID: \"c67fe06a-8f45-48d0-9451-b9157f57369e\") " pod="openshift-cluster-node-tuning-operator/tuned-92gv2"
Oct 30 21:42:35.528008 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.527943    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c67fe06a-8f45-48d0-9451-b9157f57369e-etc-kubernetes\") pod \"tuned-92gv2\" (UID: \"c67fe06a-8f45-48d0-9451-b9157f57369e\") " pod="openshift-cluster-node-tuning-operator/tuned-92gv2"
Oct 30 21:42:35.528008 ip-10-0-140-18 kubenswrapper[4473]: E1030 21:42:35.527968    4473 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ccc5af13-906a-469d-9ca5-09366d1a7e93-metrics-certs podName:ccc5af13-906a-469d-9ca5-09366d1a7e93 nodeName:}" failed. No retries permitted until 2026-10-30 21:42:36.027909662 +0000 UTC m=+1.801278147 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ccc5af13-906a-469d-9ca5-09366d1a7e93-metrics-certs") pod "network-metrics-daemon-8jvqh" (UID: "ccc5af13-906a-469d-9ca5-09366d1a7e93") : secret "metrics-daemon-secret" not found
Oct 30 21:42:35.528008 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.527976    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/2634eb5b-4ce8-4742-819a-616dcd33da39-hosts-file\") pod \"node-resolver-46nkk\" (UID: \"2634eb5b-4ce8-4742-819a-616dcd33da39\") " pod="openshift-dns/node-resolver-46nkk"
Oct 30 21:42:35.528008 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.528016    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/5de0b6c7-0d1c-4732-abce-361a46df75c3-etc-selinux\") pod \"aws-ebs-csi-driver-node-8lm5l\" (UID: \"5de0b6c7-0d1c-4732-abce-361a46df75c3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8lm5l"
Oct 30 21:42:35.529165 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.528043    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/719f98aa-6247-4973-8d2c-a91aaf276984-host-slash\") pod \"iptables-alerter-bgfld\" (UID: \"719f98aa-6247-4973-8d2c-a91aaf276984\") " pod="openshift-network-operator/iptables-alerter-bgfld"
Oct 30 21:42:35.529165 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.528068    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/641ac65e-00db-4646-b03c-9e3cb0c70feb-var-lib-openvswitch\") pod \"ovnkube-node-w6d9j\" (UID: \"641ac65e-00db-4646-b03c-9e3cb0c70feb\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6d9j"
Oct 30 21:42:35.529165 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.528094    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/641ac65e-00db-4646-b03c-9e3cb0c70feb-run-ovn\") pod \"ovnkube-node-w6d9j\" (UID: \"641ac65e-00db-4646-b03c-9e3cb0c70feb\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6d9j"
Oct 30 21:42:35.529165 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.528119    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f152360c-cf62-4d10-b2e5-b85ac7dfb739-metrics-tls\") pod \"dns-default-hbmxt\" (UID: \"f152360c-cf62-4d10-b2e5-b85ac7dfb739\") " pod="openshift-dns/dns-default-hbmxt"
Oct 30 21:42:35.529165 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.528122    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/5de0b6c7-0d1c-4732-abce-361a46df75c3-etc-selinux\") pod \"aws-ebs-csi-driver-node-8lm5l\" (UID: \"5de0b6c7-0d1c-4732-abce-361a46df75c3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8lm5l"
Oct 30 21:42:35.529165 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.528123    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/719f98aa-6247-4973-8d2c-a91aaf276984-host-slash\") pod \"iptables-alerter-bgfld\" (UID: \"719f98aa-6247-4973-8d2c-a91aaf276984\") " pod="openshift-network-operator/iptables-alerter-bgfld"
Oct 30 21:42:35.529165 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.528133    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/641ac65e-00db-4646-b03c-9e3cb0c70feb-var-lib-openvswitch\") pod \"ovnkube-node-w6d9j\" (UID: \"641ac65e-00db-4646-b03c-9e3cb0c70feb\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6d9j"
Oct 30 21:42:35.529165 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.528146    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/5de0b6c7-0d1c-4732-abce-361a46df75c3-registration-dir\") pod \"aws-ebs-csi-driver-node-8lm5l\" (UID: \"5de0b6c7-0d1c-4732-abce-361a46df75c3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8lm5l"
Oct 30 21:42:35.529165 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.528155    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/641ac65e-00db-4646-b03c-9e3cb0c70feb-run-ovn\") pod \"ovnkube-node-w6d9j\" (UID: \"641ac65e-00db-4646-b03c-9e3cb0c70feb\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6d9j"
Oct 30 21:42:35.529165 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.528171    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/03a62eeb-c318-4ce3-b6e9-1afba98920e9-multus-cni-dir\") pod \"multus-hs4mm\" (UID: \"03a62eeb-c318-4ce3-b6e9-1afba98920e9\") " pod="openshift-multus/multus-hs4mm"
Oct 30 21:42:35.529165 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.528187    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/5de0b6c7-0d1c-4732-abce-361a46df75c3-registration-dir\") pod \"aws-ebs-csi-driver-node-8lm5l\" (UID: \"5de0b6c7-0d1c-4732-abce-361a46df75c3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8lm5l"
Oct 30 21:42:35.529165 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.528196    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/03a62eeb-c318-4ce3-b6e9-1afba98920e9-host-var-lib-cni-bin\") pod \"multus-hs4mm\" (UID: \"03a62eeb-c318-4ce3-b6e9-1afba98920e9\") " pod="openshift-multus/multus-hs4mm"
Oct 30 21:42:35.529165 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.528223    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/03a62eeb-c318-4ce3-b6e9-1afba98920e9-host-run-multus-certs\") pod \"multus-hs4mm\" (UID: \"03a62eeb-c318-4ce3-b6e9-1afba98920e9\") " pod="openshift-multus/multus-hs4mm"
Oct 30 21:42:35.529165 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.528248    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/5de0b6c7-0d1c-4732-abce-361a46df75c3-kubelet-dir\") pod \"aws-ebs-csi-driver-node-8lm5l\" (UID: \"5de0b6c7-0d1c-4732-abce-361a46df75c3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8lm5l"
Oct 30 21:42:35.529165 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.528271    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/03a62eeb-c318-4ce3-b6e9-1afba98920e9-host-var-lib-cni-bin\") pod \"multus-hs4mm\" (UID: \"03a62eeb-c318-4ce3-b6e9-1afba98920e9\") " pod="openshift-multus/multus-hs4mm"
Oct 30 21:42:35.529165 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.528283    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/03a62eeb-c318-4ce3-b6e9-1afba98920e9-host-run-multus-certs\") pod \"multus-hs4mm\" (UID: \"03a62eeb-c318-4ce3-b6e9-1afba98920e9\") " pod="openshift-multus/multus-hs4mm"
Oct 30 21:42:35.529165 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.528275    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/03a62eeb-c318-4ce3-b6e9-1afba98920e9-multus-cni-dir\") pod \"multus-hs4mm\" (UID: \"03a62eeb-c318-4ce3-b6e9-1afba98920e9\") " pod="openshift-multus/multus-hs4mm"
Oct 30 21:42:35.529165 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.528274    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/5de0b6c7-0d1c-4732-abce-361a46df75c3-socket-dir\") pod \"aws-ebs-csi-driver-node-8lm5l\" (UID: \"5de0b6c7-0d1c-4732-abce-361a46df75c3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8lm5l"
Oct 30 21:42:35.529165 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.528320    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/5de0b6c7-0d1c-4732-abce-361a46df75c3-kubelet-dir\") pod \"aws-ebs-csi-driver-node-8lm5l\" (UID: \"5de0b6c7-0d1c-4732-abce-361a46df75c3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8lm5l"
Oct 30 21:42:35.529165 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.528329    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/5de0b6c7-0d1c-4732-abce-361a46df75c3-socket-dir\") pod \"aws-ebs-csi-driver-node-8lm5l\" (UID: \"5de0b6c7-0d1c-4732-abce-361a46df75c3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8lm5l"
Oct 30 21:42:35.529165 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.528332    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/641ac65e-00db-4646-b03c-9e3cb0c70feb-host-slash\") pod \"ovnkube-node-w6d9j\" (UID: \"641ac65e-00db-4646-b03c-9e3cb0c70feb\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6d9j"
Oct 30 21:42:35.529165 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.528353    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/641ac65e-00db-4646-b03c-9e3cb0c70feb-host-slash\") pod \"ovnkube-node-w6d9j\" (UID: \"641ac65e-00db-4646-b03c-9e3cb0c70feb\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6d9j"
Oct 30 21:42:35.529165 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.528368    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/641ac65e-00db-4646-b03c-9e3cb0c70feb-etc-openvswitch\") pod \"ovnkube-node-w6d9j\" (UID: \"641ac65e-00db-4646-b03c-9e3cb0c70feb\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6d9j"
Oct 30 21:42:35.529165 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.528392    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/641ac65e-00db-4646-b03c-9e3cb0c70feb-host-cni-netd\") pod \"ovnkube-node-w6d9j\" (UID: \"641ac65e-00db-4646-b03c-9e3cb0c70feb\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6d9j"
Oct 30 21:42:35.529165 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.528402    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/641ac65e-00db-4646-b03c-9e3cb0c70feb-etc-openvswitch\") pod \"ovnkube-node-w6d9j\" (UID: \"641ac65e-00db-4646-b03c-9e3cb0c70feb\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6d9j"
Oct 30 21:42:35.529165 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.528416    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c67fe06a-8f45-48d0-9451-b9157f57369e-run\") pod \"tuned-92gv2\" (UID: \"c67fe06a-8f45-48d0-9451-b9157f57369e\") " pod="openshift-cluster-node-tuning-operator/tuned-92gv2"
Oct 30 21:42:35.529165 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.528429    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/641ac65e-00db-4646-b03c-9e3cb0c70feb-host-cni-netd\") pod \"ovnkube-node-w6d9j\" (UID: \"641ac65e-00db-4646-b03c-9e3cb0c70feb\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6d9j"
Oct 30 21:42:35.529165 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.528445    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0743c782-f5ad-4d3a-86e7-fc66563902c7-cnibin\") pod \"multus-additional-cni-plugins-kkr27\" (UID: \"0743c782-f5ad-4d3a-86e7-fc66563902c7\") " pod="openshift-multus/multus-additional-cni-plugins-kkr27"
Oct 30 21:42:35.529165 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.528470    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/641ac65e-00db-4646-b03c-9e3cb0c70feb-host-kubelet\") pod \"ovnkube-node-w6d9j\" (UID: \"641ac65e-00db-4646-b03c-9e3cb0c70feb\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6d9j"
Oct 30 21:42:35.529165 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.528485    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c67fe06a-8f45-48d0-9451-b9157f57369e-run\") pod \"tuned-92gv2\" (UID: \"c67fe06a-8f45-48d0-9451-b9157f57369e\") " pod="openshift-cluster-node-tuning-operator/tuned-92gv2"
Oct 30 21:42:35.529165 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.528496    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/03a62eeb-c318-4ce3-b6e9-1afba98920e9-host-run-k8s-cni-cncf-io\") pod \"multus-hs4mm\" (UID: \"03a62eeb-c318-4ce3-b6e9-1afba98920e9\") " pod="openshift-multus/multus-hs4mm"
Oct 30 21:42:35.529165 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.528512    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0743c782-f5ad-4d3a-86e7-fc66563902c7-cnibin\") pod \"multus-additional-cni-plugins-kkr27\" (UID: \"0743c782-f5ad-4d3a-86e7-fc66563902c7\") " pod="openshift-multus/multus-additional-cni-plugins-kkr27"
Oct 30 21:42:35.529165 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.528521    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/03a62eeb-c318-4ce3-b6e9-1afba98920e9-host-var-lib-kubelet\") pod \"multus-hs4mm\" (UID: \"03a62eeb-c318-4ce3-b6e9-1afba98920e9\") " pod="openshift-multus/multus-hs4mm"
Oct 30 21:42:35.529165 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.528531    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/641ac65e-00db-4646-b03c-9e3cb0c70feb-host-kubelet\") pod \"ovnkube-node-w6d9j\" (UID: \"641ac65e-00db-4646-b03c-9e3cb0c70feb\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6d9j"
Oct 30 21:42:35.529165 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.528548    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/03a62eeb-c318-4ce3-b6e9-1afba98920e9-host-run-k8s-cni-cncf-io\") pod \"multus-hs4mm\" (UID: \"03a62eeb-c318-4ce3-b6e9-1afba98920e9\") " pod="openshift-multus/multus-hs4mm"
Oct 30 21:42:35.529165 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.528552    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0743c782-f5ad-4d3a-86e7-fc66563902c7-tuning-conf-dir\") pod \"multus-additional-cni-plugins-kkr27\" (UID: \"0743c782-f5ad-4d3a-86e7-fc66563902c7\") " pod="openshift-multus/multus-additional-cni-plugins-kkr27"
Oct 30 21:42:35.529165 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.528579    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/5de0b6c7-0d1c-4732-abce-361a46df75c3-device-dir\") pod \"aws-ebs-csi-driver-node-8lm5l\" (UID: \"5de0b6c7-0d1c-4732-abce-361a46df75c3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8lm5l"
Oct 30 21:42:35.529165 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.528584    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/03a62eeb-c318-4ce3-b6e9-1afba98920e9-host-var-lib-kubelet\") pod \"multus-hs4mm\" (UID: \"03a62eeb-c318-4ce3-b6e9-1afba98920e9\") " pod="openshift-multus/multus-hs4mm"
Oct 30 21:42:35.529165 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.528604    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/641ac65e-00db-4646-b03c-9e3cb0c70feb-host-cni-bin\") pod \"ovnkube-node-w6d9j\" (UID: \"641ac65e-00db-4646-b03c-9e3cb0c70feb\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6d9j"
Oct 30 21:42:35.529165 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.528629    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c67fe06a-8f45-48d0-9451-b9157f57369e-etc-sysctl-conf\") pod \"tuned-92gv2\" (UID: \"c67fe06a-8f45-48d0-9451-b9157f57369e\") " pod="openshift-cluster-node-tuning-operator/tuned-92gv2"
Oct 30 21:42:35.529165 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.528642    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0743c782-f5ad-4d3a-86e7-fc66563902c7-tuning-conf-dir\") pod \"multus-additional-cni-plugins-kkr27\" (UID: \"0743c782-f5ad-4d3a-86e7-fc66563902c7\") " pod="openshift-multus/multus-additional-cni-plugins-kkr27"
Oct 30 21:42:35.529165 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.528649    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/5de0b6c7-0d1c-4732-abce-361a46df75c3-device-dir\") pod \"aws-ebs-csi-driver-node-8lm5l\" (UID: \"5de0b6c7-0d1c-4732-abce-361a46df75c3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8lm5l"
Oct 30 21:42:35.529165 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.528654    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c67fe06a-8f45-48d0-9451-b9157f57369e-host\") pod \"tuned-92gv2\" (UID: \"c67fe06a-8f45-48d0-9451-b9157f57369e\") " pod="openshift-cluster-node-tuning-operator/tuned-92gv2"
Oct 30 21:42:35.529165 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.528686    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2428ae10-7385-4056-aabe-bdf1b974a36b-host\") pod \"node-ca-jwvtb\" (UID: \"2428ae10-7385-4056-aabe-bdf1b974a36b\") " pod="openshift-image-registry/node-ca-jwvtb"
Oct 30 21:42:35.529165 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.528701    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c67fe06a-8f45-48d0-9451-b9157f57369e-host\") pod \"tuned-92gv2\" (UID: \"c67fe06a-8f45-48d0-9451-b9157f57369e\") " pod="openshift-cluster-node-tuning-operator/tuned-92gv2"
Oct 30 21:42:35.529165 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.528687    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/641ac65e-00db-4646-b03c-9e3cb0c70feb-host-cni-bin\") pod \"ovnkube-node-w6d9j\" (UID: \"641ac65e-00db-4646-b03c-9e3cb0c70feb\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6d9j"
Oct 30 21:42:35.529165 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.528715    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2428ae10-7385-4056-aabe-bdf1b974a36b-host\") pod \"node-ca-jwvtb\" (UID: \"2428ae10-7385-4056-aabe-bdf1b974a36b\") " pod="openshift-image-registry/node-ca-jwvtb"
Oct 30 21:42:35.529165 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.528713    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/641ac65e-00db-4646-b03c-9e3cb0c70feb-host-run-netns\") pod \"ovnkube-node-w6d9j\" (UID: \"641ac65e-00db-4646-b03c-9e3cb0c70feb\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6d9j"
Oct 30 21:42:35.529165 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.528742    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/641ac65e-00db-4646-b03c-9e3cb0c70feb-host-run-netns\") pod \"ovnkube-node-w6d9j\" (UID: \"641ac65e-00db-4646-b03c-9e3cb0c70feb\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6d9j"
Oct 30 21:42:35.529165 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.528752    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/0032cb99-3a37-40c6-98cb-2d40ae4a9319-dbus\") pod \"global-pull-secret-syncer-dt7cq\" (UID: \"0032cb99-3a37-40c6-98cb-2d40ae4a9319\") " pod="kube-system/global-pull-secret-syncer-dt7cq"
Oct 30 21:42:35.529165 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.528764    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c67fe06a-8f45-48d0-9451-b9157f57369e-etc-sysctl-conf\") pod \"tuned-92gv2\" (UID: \"c67fe06a-8f45-48d0-9451-b9157f57369e\") " pod="openshift-cluster-node-tuning-operator/tuned-92gv2"
Oct 30 21:42:35.529165 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.528781    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/03a62eeb-c318-4ce3-b6e9-1afba98920e9-multus-socket-dir-parent\") pod \"multus-hs4mm\" (UID: \"03a62eeb-c318-4ce3-b6e9-1afba98920e9\") " pod="openshift-multus/multus-hs4mm"
Oct 30 21:42:35.529165 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.528805    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/03a62eeb-c318-4ce3-b6e9-1afba98920e9-hostroot\") pod \"multus-hs4mm\" (UID: \"03a62eeb-c318-4ce3-b6e9-1afba98920e9\") " pod="openshift-multus/multus-hs4mm"
Oct 30 21:42:35.529165 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.528832    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0743c782-f5ad-4d3a-86e7-fc66563902c7-system-cni-dir\") pod \"multus-additional-cni-plugins-kkr27\" (UID: \"0743c782-f5ad-4d3a-86e7-fc66563902c7\") " pod="openshift-multus/multus-additional-cni-plugins-kkr27"
Oct 30 21:42:35.529165 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.528846    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/03a62eeb-c318-4ce3-b6e9-1afba98920e9-hostroot\") pod \"multus-hs4mm\" (UID: \"03a62eeb-c318-4ce3-b6e9-1afba98920e9\") " pod="openshift-multus/multus-hs4mm"
Oct 30 21:42:35.529165 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.528849    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/03a62eeb-c318-4ce3-b6e9-1afba98920e9-multus-socket-dir-parent\") pod \"multus-hs4mm\" (UID: \"03a62eeb-c318-4ce3-b6e9-1afba98920e9\") " pod="openshift-multus/multus-hs4mm"
Oct 30 21:42:35.529165 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.528851    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/0032cb99-3a37-40c6-98cb-2d40ae4a9319-dbus\") pod \"global-pull-secret-syncer-dt7cq\" (UID: \"0032cb99-3a37-40c6-98cb-2d40ae4a9319\") " pod="kube-system/global-pull-secret-syncer-dt7cq"
Oct 30 21:42:35.529165 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.528861    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/641ac65e-00db-4646-b03c-9e3cb0c70feb-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-w6d9j\" (UID: \"641ac65e-00db-4646-b03c-9e3cb0c70feb\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6d9j"
Oct 30 21:42:35.529165 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.528875    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0743c782-f5ad-4d3a-86e7-fc66563902c7-system-cni-dir\") pod \"multus-additional-cni-plugins-kkr27\" (UID: \"0743c782-f5ad-4d3a-86e7-fc66563902c7\") " pod="openshift-multus/multus-additional-cni-plugins-kkr27"
Oct 30 21:42:35.529165 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.528884    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/641ac65e-00db-4646-b03c-9e3cb0c70feb-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-w6d9j\" (UID: \"641ac65e-00db-4646-b03c-9e3cb0c70feb\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6d9j"
Oct 30 21:42:35.529165 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.528886    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c67fe06a-8f45-48d0-9451-b9157f57369e-etc-sysconfig\") pod \"tuned-92gv2\" (UID: \"c67fe06a-8f45-48d0-9451-b9157f57369e\") " pod="openshift-cluster-node-tuning-operator/tuned-92gv2"
Oct 30 21:42:35.529165 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.528912    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c67fe06a-8f45-48d0-9451-b9157f57369e-var-lib-kubelet\") pod \"tuned-92gv2\" (UID: \"c67fe06a-8f45-48d0-9451-b9157f57369e\") " pod="openshift-cluster-node-tuning-operator/tuned-92gv2"
Oct 30 21:42:35.529165 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.528962    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/32a1c660-effa-4b6c-975a-dbb68c468eda-crio-socket\") pod \"insights-runtime-extractor-fncvg\" (UID: \"32a1c660-effa-4b6c-975a-dbb68c468eda\") " pod="openshift-insights/insights-runtime-extractor-fncvg"
Oct 30 21:42:35.529165 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.528990    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c67fe06a-8f45-48d0-9451-b9157f57369e-var-lib-kubelet\") pod \"tuned-92gv2\" (UID: \"c67fe06a-8f45-48d0-9451-b9157f57369e\") " pod="openshift-cluster-node-tuning-operator/tuned-92gv2"
Oct 30 21:42:35.529165 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.528988    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/32a1c660-effa-4b6c-975a-dbb68c468eda-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fncvg\" (UID: \"32a1c660-effa-4b6c-975a-dbb68c468eda\") " pod="openshift-insights/insights-runtime-extractor-fncvg"
Oct 30 21:42:35.529165 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.529031    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/641ac65e-00db-4646-b03c-9e3cb0c70feb-run-systemd\") pod \"ovnkube-node-w6d9j\" (UID: \"641ac65e-00db-4646-b03c-9e3cb0c70feb\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6d9j"
Oct 30 21:42:35.529165 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.529060    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/641ac65e-00db-4646-b03c-9e3cb0c70feb-node-log\") pod \"ovnkube-node-w6d9j\" (UID: \"641ac65e-00db-4646-b03c-9e3cb0c70feb\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6d9j"
Oct 30 21:42:35.529165 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.529069    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/641ac65e-00db-4646-b03c-9e3cb0c70feb-run-systemd\") pod \"ovnkube-node-w6d9j\" (UID: \"641ac65e-00db-4646-b03c-9e3cb0c70feb\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6d9j"
Oct 30 21:42:35.529165 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.528961    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c67fe06a-8f45-48d0-9451-b9157f57369e-etc-sysconfig\") pod \"tuned-92gv2\" (UID: \"c67fe06a-8f45-48d0-9451-b9157f57369e\") " pod="openshift-cluster-node-tuning-operator/tuned-92gv2"
Oct 30 21:42:35.529165 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.529032    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/32a1c660-effa-4b6c-975a-dbb68c468eda-crio-socket\") pod \"insights-runtime-extractor-fncvg\" (UID: \"32a1c660-effa-4b6c-975a-dbb68c468eda\") " pod="openshift-insights/insights-runtime-extractor-fncvg"
Oct 30 21:42:35.529165 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.529086    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c67fe06a-8f45-48d0-9451-b9157f57369e-lib-modules\") pod \"tuned-92gv2\" (UID: \"c67fe06a-8f45-48d0-9451-b9157f57369e\") " pod="openshift-cluster-node-tuning-operator/tuned-92gv2"
Oct 30 21:42:35.529165 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.529095    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/641ac65e-00db-4646-b03c-9e3cb0c70feb-node-log\") pod \"ovnkube-node-w6d9j\" (UID: \"641ac65e-00db-4646-b03c-9e3cb0c70feb\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6d9j"
Oct 30 21:42:35.529165 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.529116    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/03a62eeb-c318-4ce3-b6e9-1afba98920e9-system-cni-dir\") pod \"multus-hs4mm\" (UID: \"03a62eeb-c318-4ce3-b6e9-1afba98920e9\") " pod="openshift-multus/multus-hs4mm"
Oct 30 21:42:35.529165 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.529146    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/641ac65e-00db-4646-b03c-9e3cb0c70feb-systemd-units\") pod \"ovnkube-node-w6d9j\" (UID: \"641ac65e-00db-4646-b03c-9e3cb0c70feb\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6d9j"
Oct 30 21:42:35.529165 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.529156    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c67fe06a-8f45-48d0-9451-b9157f57369e-lib-modules\") pod \"tuned-92gv2\" (UID: \"c67fe06a-8f45-48d0-9451-b9157f57369e\") " pod="openshift-cluster-node-tuning-operator/tuned-92gv2"
Oct 30 21:42:35.529165 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.529168    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c67fe06a-8f45-48d0-9451-b9157f57369e-etc-sysctl-d\") pod \"tuned-92gv2\" (UID: \"c67fe06a-8f45-48d0-9451-b9157f57369e\") " pod="openshift-cluster-node-tuning-operator/tuned-92gv2"
Oct 30 21:42:35.529165 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.529196    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/641ac65e-00db-4646-b03c-9e3cb0c70feb-systemd-units\") pod \"ovnkube-node-w6d9j\" (UID: \"641ac65e-00db-4646-b03c-9e3cb0c70feb\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6d9j"
Oct 30 21:42:35.529165 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.529200    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/03a62eeb-c318-4ce3-b6e9-1afba98920e9-system-cni-dir\") pod \"multus-hs4mm\" (UID: \"03a62eeb-c318-4ce3-b6e9-1afba98920e9\") " pod="openshift-multus/multus-hs4mm"
Oct 30 21:42:35.529165 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.529195    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/03a62eeb-c318-4ce3-b6e9-1afba98920e9-multus-conf-dir\") pod \"multus-hs4mm\" (UID: \"03a62eeb-c318-4ce3-b6e9-1afba98920e9\") " pod="openshift-multus/multus-hs4mm"
Oct 30 21:42:35.531296 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.529232    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/03a62eeb-c318-4ce3-b6e9-1afba98920e9-multus-conf-dir\") pod \"multus-hs4mm\" (UID: \"03a62eeb-c318-4ce3-b6e9-1afba98920e9\") " pod="openshift-multus/multus-hs4mm"
Oct 30 21:42:35.531296 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.529274    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c67fe06a-8f45-48d0-9451-b9157f57369e-etc-sysctl-d\") pod \"tuned-92gv2\" (UID: \"c67fe06a-8f45-48d0-9451-b9157f57369e\") " pod="openshift-cluster-node-tuning-operator/tuned-92gv2"
Oct 30 21:42:35.566059 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.565984    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Oct 30 21:42:35.566059 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.565999    4473 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Oct 30 21:42:35.566059 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.566043    4473 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Oct 30 21:42:35.566233 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.566070    4473 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Oct 30 21:42:35.566233 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.566108    4473 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Oct 30 21:42:35.566233 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.566157    4473 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Oct 30 21:42:35.566393 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.566379    4473 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Oct 30 21:42:35.566442 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.566403    4473 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Oct 30 21:42:35.566488 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.566474    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-qdnct\""
Oct 30 21:42:35.566549 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.566536    4473 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Oct 30 21:42:35.566651 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.566634    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Oct 30 21:42:35.566781 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.566766    4473 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Oct 30 21:42:35.566871 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.566856    4473 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Oct 30 21:42:35.566909 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.566860    4473 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Oct 30 21:42:35.567117 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.567102    4473 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Oct 30 21:42:35.567178 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.567121    4473 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Oct 30 21:42:35.567222 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.567176    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Oct 30 21:42:35.567222 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.567185    4473 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Oct 30 21:42:35.567222 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.567188    4473 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Oct 30 21:42:35.567357 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.567298    4473 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Oct 30 21:42:35.567357 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.567297    4473 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Oct 30 21:42:35.567357 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.567336    4473 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Oct 30 21:42:35.567357 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.567349    4473 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Oct 30 21:42:35.567536 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.567364    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-xw9dh\""
Oct 30 21:42:35.567536 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.567402    4473 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Oct 30 21:42:35.567536 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.567455    4473 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Oct 30 21:42:35.567536 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.567465    4473 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Oct 30 21:42:35.567536 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.567476    4473 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Oct 30 21:42:35.567536 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.567529    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-5cjr4\""
Oct 30 21:42:35.567760 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.567549    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Oct 30 21:42:35.567760 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.567571    4473 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Oct 30 21:42:35.567760 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.567584    4473 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Oct 30 21:42:35.567760 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.567604    4473 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Oct 30 21:42:35.567760 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.567605    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-fxktb\""
Oct 30 21:42:35.567760 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.567666    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-dcz6k\""
Oct 30 21:42:35.567760 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.567676    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-2pvwl\""
Oct 30 21:42:35.567760 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.567685    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-6vmmq\""
Oct 30 21:42:35.567760 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.567676    4473 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Oct 30 21:42:35.567760 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.567713    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-rfsrz\""
Oct 30 21:42:35.567760 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.567734    4473 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Oct 30 21:42:35.568093 ip-10-0-140-18 kubenswrapper[4473]: E1030 21:42:35.567783    4473 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 30 21:42:35.568093 ip-10-0-140-18 kubenswrapper[4473]: E1030 21:42:35.567845    4473 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/32a1c660-effa-4b6c-975a-dbb68c468eda-insights-runtime-extractor-tls podName:32a1c660-effa-4b6c-975a-dbb68c468eda nodeName:}" failed. No retries permitted until 2026-10-30 21:42:36.067829434 +0000 UTC m=+1.841197898 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/32a1c660-effa-4b6c-975a-dbb68c468eda-insights-runtime-extractor-tls") pod "insights-runtime-extractor-fncvg" (UID: "32a1c660-effa-4b6c-975a-dbb68c468eda") : secret "insights-runtime-extractor-tls" not found
Oct 30 21:42:35.569627 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.569607    4473 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Oct 30 21:42:35.569698 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.569645    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Oct 30 21:42:35.569698 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.569690    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-tpd8h\""
Oct 30 21:42:35.569777 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.569701    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-9lfqr\""
Oct 30 21:42:35.569777 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.569703    4473 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Oct 30 21:42:35.569777 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.569704    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-gbz5v\""
Oct 30 21:42:35.569930 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.569902    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/32a1c660-effa-4b6c-975a-dbb68c468eda-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fncvg\" (UID: \"32a1c660-effa-4b6c-975a-dbb68c468eda\") " pod="openshift-insights/insights-runtime-extractor-fncvg"
Oct 30 21:42:35.569969 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.569944    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pn9mk\" (UniqueName: \"kubernetes.io/projected/719f98aa-6247-4973-8d2c-a91aaf276984-kube-api-access-pn9mk\") pod \"iptables-alerter-bgfld\" (UID: \"719f98aa-6247-4973-8d2c-a91aaf276984\") " pod="openshift-network-operator/iptables-alerter-bgfld"
Oct 30 21:42:35.569998 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.569974    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rfxjl\" (UniqueName: \"kubernetes.io/projected/5de0b6c7-0d1c-4732-abce-361a46df75c3-kube-api-access-rfxjl\") pod \"aws-ebs-csi-driver-node-8lm5l\" (UID: \"5de0b6c7-0d1c-4732-abce-361a46df75c3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8lm5l"
Oct 30 21:42:35.571307 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.571291    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7vks4\" (UniqueName: \"kubernetes.io/projected/4436ec50-e190-4c1c-8b6c-2b9b5d12cb9f-kube-api-access-7vks4\") pod \"ingress-canary-5m6cr\" (UID: \"4436ec50-e190-4c1c-8b6c-2b9b5d12cb9f\") " pod="openshift-ingress-canary/ingress-canary-5m6cr"
Oct 30 21:42:35.571361 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.571340    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-htnvw\" (UniqueName: \"kubernetes.io/projected/2634eb5b-4ce8-4742-819a-616dcd33da39-kube-api-access-htnvw\") pod \"node-resolver-46nkk\" (UID: \"2634eb5b-4ce8-4742-819a-616dcd33da39\") " pod="openshift-dns/node-resolver-46nkk"
Oct 30 21:42:35.571399 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.571341    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5xlxd\" (UniqueName: \"kubernetes.io/projected/2428ae10-7385-4056-aabe-bdf1b974a36b-kube-api-access-5xlxd\") pod \"node-ca-jwvtb\" (UID: \"2428ae10-7385-4056-aabe-bdf1b974a36b\") " pod="openshift-image-registry/node-ca-jwvtb"
Oct 30 21:42:35.571399 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.571378    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n6qcb\" (UniqueName: \"kubernetes.io/projected/f152360c-cf62-4d10-b2e5-b85ac7dfb739-kube-api-access-n6qcb\") pod \"dns-default-hbmxt\" (UID: \"f152360c-cf62-4d10-b2e5-b85ac7dfb739\") " pod="openshift-dns/dns-default-hbmxt"
Oct 30 21:42:35.571481 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.571466    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-js6zb\" (UniqueName: \"kubernetes.io/projected/10d2e5ae-c1c4-400c-8c67-3aebb6caaa6a-kube-api-access-js6zb\") pod \"network-check-target-9st74\" (UID: \"10d2e5ae-c1c4-400c-8c67-3aebb6caaa6a\") " pod="openshift-network-diagnostics/network-check-target-9st74"
Oct 30 21:42:35.573111 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.573091    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g98k9\" (UniqueName: \"kubernetes.io/projected/525a0437-9a44-425b-966a-b118d91a4522-kube-api-access-g98k9\") pod \"migrator-66df5d496b-gj8rz\" (UID: \"525a0437-9a44-425b-966a-b118d91a4522\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-gj8rz"
Oct 30 21:42:35.573194 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.573096    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zzqrt\" (UniqueName: \"kubernetes.io/projected/32a1c660-effa-4b6c-975a-dbb68c468eda-kube-api-access-zzqrt\") pod \"insights-runtime-extractor-fncvg\" (UID: \"32a1c660-effa-4b6c-975a-dbb68c468eda\") " pod="openshift-insights/insights-runtime-extractor-fncvg"
Oct 30 21:42:35.573289 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.573266    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-64s8x\" (UniqueName: \"kubernetes.io/projected/c67fe06a-8f45-48d0-9451-b9157f57369e-kube-api-access-64s8x\") pod \"tuned-92gv2\" (UID: \"c67fe06a-8f45-48d0-9451-b9157f57369e\") " pod="openshift-cluster-node-tuning-operator/tuned-92gv2"
Oct 30 21:42:35.573402 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.573387    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lnn4t\" (UniqueName: \"kubernetes.io/projected/bddede0e-ee07-4344-878d-10b967435e13-kube-api-access-lnn4t\") pod \"periodic-gathering-bcmkf-fm7wg\" (UID: \"bddede0e-ee07-4344-878d-10b967435e13\") " pod="openshift-insights/periodic-gathering-bcmkf-fm7wg"
Oct 30 21:42:35.573541 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.573519    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9hm6j\" (UniqueName: \"kubernetes.io/projected/641ac65e-00db-4646-b03c-9e3cb0c70feb-kube-api-access-9hm6j\") pod \"ovnkube-node-w6d9j\" (UID: \"641ac65e-00db-4646-b03c-9e3cb0c70feb\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6d9j"
Oct 30 21:42:35.574999 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.574983    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/0743c782-f5ad-4d3a-86e7-fc66563902c7-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-kkr27\" (UID: \"0743c782-f5ad-4d3a-86e7-fc66563902c7\") " pod="openshift-multus/multus-additional-cni-plugins-kkr27"
Oct 30 21:42:35.575087 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.574986    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/641ac65e-00db-4646-b03c-9e3cb0c70feb-env-overrides\") pod \"ovnkube-node-w6d9j\" (UID: \"641ac65e-00db-4646-b03c-9e3cb0c70feb\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6d9j"
Oct 30 21:42:35.575087 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.575007    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bddede0e-ee07-4344-878d-10b967435e13-service-ca-bundle\") pod \"periodic-gathering-bcmkf-fm7wg\" (UID: \"bddede0e-ee07-4344-878d-10b967435e13\") " pod="openshift-insights/periodic-gathering-bcmkf-fm7wg"
Oct 30 21:42:35.575087 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.575007    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/2428ae10-7385-4056-aabe-bdf1b974a36b-serviceca\") pod \"node-ca-jwvtb\" (UID: \"2428ae10-7385-4056-aabe-bdf1b974a36b\") " pod="openshift-image-registry/node-ca-jwvtb"
Oct 30 21:42:35.576230 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.576209    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/641ac65e-00db-4646-b03c-9e3cb0c70feb-ovn-node-metrics-cert\") pod \"ovnkube-node-w6d9j\" (UID: \"641ac65e-00db-4646-b03c-9e3cb0c70feb\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6d9j"
Oct 30 21:42:35.576293 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.576239    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/03a62eeb-c318-4ce3-b6e9-1afba98920e9-cni-binary-copy\") pod \"multus-hs4mm\" (UID: \"03a62eeb-c318-4ce3-b6e9-1afba98920e9\") " pod="openshift-multus/multus-hs4mm"
Oct 30 21:42:35.576293 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.576249    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/641ac65e-00db-4646-b03c-9e3cb0c70feb-ovnkube-config\") pod \"ovnkube-node-w6d9j\" (UID: \"641ac65e-00db-4646-b03c-9e3cb0c70feb\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6d9j"
Oct 30 21:42:35.576293 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.576267    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0743c782-f5ad-4d3a-86e7-fc66563902c7-cni-binary-copy\") pod \"multus-additional-cni-plugins-kkr27\" (UID: \"0743c782-f5ad-4d3a-86e7-fc66563902c7\") " pod="openshift-multus/multus-additional-cni-plugins-kkr27"
Oct 30 21:42:35.576293 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.576281    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4436ec50-e190-4c1c-8b6c-2b9b5d12cb9f-cert\") pod \"ingress-canary-5m6cr\" (UID: \"4436ec50-e190-4c1c-8b6c-2b9b5d12cb9f\") " pod="openshift-ingress-canary/ingress-canary-5m6cr"
Oct 30 21:42:35.576293 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.576288    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/f152360c-cf62-4d10-b2e5-b85ac7dfb739-config-volume\") pod \"dns-default-hbmxt\" (UID: \"f152360c-cf62-4d10-b2e5-b85ac7dfb739\") " pod="openshift-dns/dns-default-hbmxt"
Oct 30 21:42:35.577526 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.577512    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/0743c782-f5ad-4d3a-86e7-fc66563902c7-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-kkr27\" (UID: \"0743c782-f5ad-4d3a-86e7-fc66563902c7\") " pod="openshift-multus/multus-additional-cni-plugins-kkr27"
Oct 30 21:42:35.577569 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.577534    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/03a62eeb-c318-4ce3-b6e9-1afba98920e9-multus-daemon-config\") pod \"multus-hs4mm\" (UID: \"03a62eeb-c318-4ce3-b6e9-1afba98920e9\") " pod="openshift-multus/multus-hs4mm"
Oct 30 21:42:35.578500 ip-10-0-140-18 kubenswrapper[4473]: E1030 21:42:35.578485    4473 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 30 21:42:35.578561 ip-10-0-140-18 kubenswrapper[4473]: E1030 21:42:35.578549    4473 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f152360c-cf62-4d10-b2e5-b85ac7dfb739-metrics-tls podName:f152360c-cf62-4d10-b2e5-b85ac7dfb739 nodeName:}" failed. No retries permitted until 2026-10-30 21:42:36.078531532 +0000 UTC m=+1.851899997 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f152360c-cf62-4d10-b2e5-b85ac7dfb739-metrics-tls") pod "dns-default-hbmxt" (UID: "f152360c-cf62-4d10-b2e5-b85ac7dfb739") : secret "dns-default-metrics-tls" not found
Oct 30 21:42:35.616111 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.616078    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-qtzdm\""
Oct 30 21:42:35.616111 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.616079    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-zpjmc\""
Oct 30 21:42:35.616660 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.616641    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-6j5v7\""
Oct 30 21:42:35.616719 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.616656    4473 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Oct 30 21:42:35.616719 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.616663    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-msl6j\""
Oct 30 21:42:35.616907 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.616894    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Oct 30 21:42:35.616989 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.616978    4473 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Oct 30 21:42:35.618610 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.618591    4473 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Oct 30 21:42:35.624610 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.624591    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/641ac65e-00db-4646-b03c-9e3cb0c70feb-ovnkube-script-lib\") pod \"ovnkube-node-w6d9j\" (UID: \"641ac65e-00db-4646-b03c-9e3cb0c70feb\") " pod="openshift-ovn-kubernetes/ovnkube-node-w6d9j"
Oct 30 21:42:35.625806 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.625790    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bddede0e-ee07-4344-878d-10b967435e13-trusted-ca-bundle\") pod \"periodic-gathering-bcmkf-fm7wg\" (UID: \"bddede0e-ee07-4344-878d-10b967435e13\") " pod="openshift-insights/periodic-gathering-bcmkf-fm7wg"
Oct 30 21:42:35.626063 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.626048    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0032cb99-3a37-40c6-98cb-2d40ae4a9319-original-pull-secret\") pod \"global-pull-secret-syncer-dt7cq\" (UID: \"0032cb99-3a37-40c6-98cb-2d40ae4a9319\") " pod="kube-system/global-pull-secret-syncer-dt7cq"
Oct 30 21:42:35.626825 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.626808    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/719f98aa-6247-4973-8d2c-a91aaf276984-iptables-alerter-script\") pod \"iptables-alerter-bgfld\" (UID: \"719f98aa-6247-4973-8d2c-a91aaf276984\") " pod="openshift-network-operator/iptables-alerter-bgfld"
Oct 30 21:42:35.651872 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:35.651854    4473 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-46nkk_2634eb5b-4ce8-4742-819a-616dcd33da39/dns-node-resolver/0.log"
Oct 30 21:42:36.032865 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:36.032828    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ccc5af13-906a-469d-9ca5-09366d1a7e93-metrics-certs\") pod \"network-metrics-daemon-8jvqh\" (UID: \"ccc5af13-906a-469d-9ca5-09366d1a7e93\") " pod="openshift-multus/network-metrics-daemon-8jvqh"
Oct 30 21:42:36.033195 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:36.033178    4473 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Oct 30 21:42:36.035416 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:36.035388    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ccc5af13-906a-469d-9ca5-09366d1a7e93-metrics-certs\") pod \"network-metrics-daemon-8jvqh\" (UID: \"ccc5af13-906a-469d-9ca5-09366d1a7e93\") " pod="openshift-multus/network-metrics-daemon-8jvqh"
Oct 30 21:42:36.133140 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:36.133111    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f152360c-cf62-4d10-b2e5-b85ac7dfb739-metrics-tls\") pod \"dns-default-hbmxt\" (UID: \"f152360c-cf62-4d10-b2e5-b85ac7dfb739\") " pod="openshift-dns/dns-default-hbmxt"
Oct 30 21:42:36.133287 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:36.133168    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/32a1c660-effa-4b6c-975a-dbb68c468eda-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fncvg\" (UID: \"32a1c660-effa-4b6c-975a-dbb68c468eda\") " pod="openshift-insights/insights-runtime-extractor-fncvg"
Oct 30 21:42:36.133287 ip-10-0-140-18 kubenswrapper[4473]: E1030 21:42:36.133254    4473 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 30 21:42:36.133287 ip-10-0-140-18 kubenswrapper[4473]: E1030 21:42:36.133256    4473 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 30 21:42:36.133410 ip-10-0-140-18 kubenswrapper[4473]: E1030 21:42:36.133301    4473 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/32a1c660-effa-4b6c-975a-dbb68c468eda-insights-runtime-extractor-tls podName:32a1c660-effa-4b6c-975a-dbb68c468eda nodeName:}" failed. No retries permitted until 2026-10-30 21:42:37.133289135 +0000 UTC m=+2.906657613 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/32a1c660-effa-4b6c-975a-dbb68c468eda-insights-runtime-extractor-tls") pod "insights-runtime-extractor-fncvg" (UID: "32a1c660-effa-4b6c-975a-dbb68c468eda") : secret "insights-runtime-extractor-tls" not found
Oct 30 21:42:36.133410 ip-10-0-140-18 kubenswrapper[4473]: E1030 21:42:36.133318    4473 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f152360c-cf62-4d10-b2e5-b85ac7dfb739-metrics-tls podName:f152360c-cf62-4d10-b2e5-b85ac7dfb739 nodeName:}" failed. No retries permitted until 2026-10-30 21:42:37.133306129 +0000 UTC m=+2.906674597 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f152360c-cf62-4d10-b2e5-b85ac7dfb739-metrics-tls") pod "dns-default-hbmxt" (UID: "f152360c-cf62-4d10-b2e5-b85ac7dfb739") : secret "dns-default-metrics-tls" not found
Oct 30 21:42:36.313029 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:36.312956    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8jvqh"
Oct 30 21:42:36.771628 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:36.771597    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-8jvqh"]
Oct 30 21:42:36.774675 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:36.774646    4473 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podccc5af13_906a_469d_9ca5_09366d1a7e93.slice/crio-644bb56e30e784c7907e01054473a854c1ff4354c687e130ccd6d9f1c4068222 WatchSource:0}: Error finding container 644bb56e30e784c7907e01054473a854c1ff4354c687e130ccd6d9f1c4068222: Status 404 returned error can't find the container with id 644bb56e30e784c7907e01054473a854c1ff4354c687e130ccd6d9f1c4068222
Oct 30 21:42:36.776793 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:36.776776    4473 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 21:42:36.853714 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:36.853640    4473 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-jwvtb_2428ae10-7385-4056-aabe-bdf1b974a36b/node-ca/0.log"
Oct 30 21:42:36.957314 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:36.957290    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-w6d9j"
Oct 30 21:42:37.141595 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:37.141564    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f152360c-cf62-4d10-b2e5-b85ac7dfb739-metrics-tls\") pod \"dns-default-hbmxt\" (UID: \"f152360c-cf62-4d10-b2e5-b85ac7dfb739\") " pod="openshift-dns/dns-default-hbmxt"
Oct 30 21:42:37.141820 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:37.141620    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/32a1c660-effa-4b6c-975a-dbb68c468eda-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fncvg\" (UID: \"32a1c660-effa-4b6c-975a-dbb68c468eda\") " pod="openshift-insights/insights-runtime-extractor-fncvg"
Oct 30 21:42:37.141820 ip-10-0-140-18 kubenswrapper[4473]: E1030 21:42:37.141735    4473 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 30 21:42:37.141820 ip-10-0-140-18 kubenswrapper[4473]: E1030 21:42:37.141765    4473 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 30 21:42:37.141820 ip-10-0-140-18 kubenswrapper[4473]: E1030 21:42:37.141795    4473 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/32a1c660-effa-4b6c-975a-dbb68c468eda-insights-runtime-extractor-tls podName:32a1c660-effa-4b6c-975a-dbb68c468eda nodeName:}" failed. No retries permitted until 2026-10-30 21:42:39.141781982 +0000 UTC m=+4.915150446 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/32a1c660-effa-4b6c-975a-dbb68c468eda-insights-runtime-extractor-tls") pod "insights-runtime-extractor-fncvg" (UID: "32a1c660-effa-4b6c-975a-dbb68c468eda") : secret "insights-runtime-extractor-tls" not found
Oct 30 21:42:37.141820 ip-10-0-140-18 kubenswrapper[4473]: E1030 21:42:37.141848    4473 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f152360c-cf62-4d10-b2e5-b85ac7dfb739-metrics-tls podName:f152360c-cf62-4d10-b2e5-b85ac7dfb739 nodeName:}" failed. No retries permitted until 2026-10-30 21:42:39.141830599 +0000 UTC m=+4.915199068 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f152360c-cf62-4d10-b2e5-b85ac7dfb739-metrics-tls") pod "dns-default-hbmxt" (UID: "f152360c-cf62-4d10-b2e5-b85ac7dfb739") : secret "dns-default-metrics-tls" not found
Oct 30 21:42:37.397262 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:37.397184    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-8jvqh" event={"ID":"ccc5af13-906a-469d-9ca5-09366d1a7e93","Type":"ContainerStarted","Data":"644bb56e30e784c7907e01054473a854c1ff4354c687e130ccd6d9f1c4068222"}
Oct 30 21:42:37.452128 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:37.452096    4473 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-5m6cr_4436ec50-e190-4c1c-8b6c-2b9b5d12cb9f/serve-healthcheck-canary/0.log"
Oct 30 21:42:38.252642 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:38.252611    4473 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-66df5d496b-gj8rz_525a0437-9a44-425b-966a-b118d91a4522/migrator/0.log"
Oct 30 21:42:39.154142 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:39.154108    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/32a1c660-effa-4b6c-975a-dbb68c468eda-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fncvg\" (UID: \"32a1c660-effa-4b6c-975a-dbb68c468eda\") " pod="openshift-insights/insights-runtime-extractor-fncvg"
Oct 30 21:42:39.154545 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:39.154180    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f152360c-cf62-4d10-b2e5-b85ac7dfb739-metrics-tls\") pod \"dns-default-hbmxt\" (UID: \"f152360c-cf62-4d10-b2e5-b85ac7dfb739\") " pod="openshift-dns/dns-default-hbmxt"
Oct 30 21:42:39.154545 ip-10-0-140-18 kubenswrapper[4473]: E1030 21:42:39.154278    4473 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 30 21:42:39.154545 ip-10-0-140-18 kubenswrapper[4473]: E1030 21:42:39.154333    4473 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/32a1c660-effa-4b6c-975a-dbb68c468eda-insights-runtime-extractor-tls podName:32a1c660-effa-4b6c-975a-dbb68c468eda nodeName:}" failed. No retries permitted until 2026-10-30 21:42:43.154320501 +0000 UTC m=+8.927688966 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/32a1c660-effa-4b6c-975a-dbb68c468eda-insights-runtime-extractor-tls") pod "insights-runtime-extractor-fncvg" (UID: "32a1c660-effa-4b6c-975a-dbb68c468eda") : secret "insights-runtime-extractor-tls" not found
Oct 30 21:42:39.154545 ip-10-0-140-18 kubenswrapper[4473]: E1030 21:42:39.154340    4473 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 30 21:42:39.154545 ip-10-0-140-18 kubenswrapper[4473]: E1030 21:42:39.154387    4473 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f152360c-cf62-4d10-b2e5-b85ac7dfb739-metrics-tls podName:f152360c-cf62-4d10-b2e5-b85ac7dfb739 nodeName:}" failed. No retries permitted until 2026-10-30 21:42:43.154375371 +0000 UTC m=+8.927743837 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f152360c-cf62-4d10-b2e5-b85ac7dfb739-metrics-tls") pod "dns-default-hbmxt" (UID: "f152360c-cf62-4d10-b2e5-b85ac7dfb739") : secret "dns-default-metrics-tls" not found
Oct 30 21:42:39.406071 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:39.405988    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-8jvqh" event={"ID":"ccc5af13-906a-469d-9ca5-09366d1a7e93","Type":"ContainerStarted","Data":"4fe8065fa058f155a7946da3ef55731837ccda0c56a9ebba530f2055eaa62253"}
Oct 30 21:42:39.406388 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:39.406366    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-8jvqh" event={"ID":"ccc5af13-906a-469d-9ca5-09366d1a7e93","Type":"ContainerStarted","Data":"342f852d70c3b6e3d1248c29ade73a24839bf95de8502dd6f0990dc73205bb3c"}
Oct 30 21:42:43.177889 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:43.177842    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/32a1c660-effa-4b6c-975a-dbb68c468eda-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fncvg\" (UID: \"32a1c660-effa-4b6c-975a-dbb68c468eda\") " pod="openshift-insights/insights-runtime-extractor-fncvg"
Oct 30 21:42:43.178371 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:43.177902    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f152360c-cf62-4d10-b2e5-b85ac7dfb739-metrics-tls\") pod \"dns-default-hbmxt\" (UID: \"f152360c-cf62-4d10-b2e5-b85ac7dfb739\") " pod="openshift-dns/dns-default-hbmxt"
Oct 30 21:42:43.178371 ip-10-0-140-18 kubenswrapper[4473]: E1030 21:42:43.178042    4473 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 30 21:42:43.178371 ip-10-0-140-18 kubenswrapper[4473]: E1030 21:42:43.178042    4473 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 30 21:42:43.178371 ip-10-0-140-18 kubenswrapper[4473]: E1030 21:42:43.178120    4473 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f152360c-cf62-4d10-b2e5-b85ac7dfb739-metrics-tls podName:f152360c-cf62-4d10-b2e5-b85ac7dfb739 nodeName:}" failed. No retries permitted until 2026-10-30 21:42:51.178106925 +0000 UTC m=+16.951475415 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f152360c-cf62-4d10-b2e5-b85ac7dfb739-metrics-tls") pod "dns-default-hbmxt" (UID: "f152360c-cf62-4d10-b2e5-b85ac7dfb739") : secret "dns-default-metrics-tls" not found
Oct 30 21:42:43.178371 ip-10-0-140-18 kubenswrapper[4473]: E1030 21:42:43.178134    4473 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/32a1c660-effa-4b6c-975a-dbb68c468eda-insights-runtime-extractor-tls podName:32a1c660-effa-4b6c-975a-dbb68c468eda nodeName:}" failed. No retries permitted until 2026-10-30 21:42:51.178129032 +0000 UTC m=+16.951497510 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/32a1c660-effa-4b6c-975a-dbb68c468eda-insights-runtime-extractor-tls") pod "insights-runtime-extractor-fncvg" (UID: "32a1c660-effa-4b6c-975a-dbb68c468eda") : secret "insights-runtime-extractor-tls" not found
Oct 30 21:42:44.808422 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:44.808390    4473 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-140-18.us-west-1.compute.internal" event="NodeReady"
Oct 30 21:42:45.174879 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:45.174845    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-w6d9j"
Oct 30 21:42:45.175131 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:45.175116    4473 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 30 21:42:45.403891 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:45.403866    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-w6d9j"
Oct 30 21:42:45.621133 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:45.621107    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-9st74"
Oct 30 21:42:51.235760 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:51.235707    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/32a1c660-effa-4b6c-975a-dbb68c468eda-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fncvg\" (UID: \"32a1c660-effa-4b6c-975a-dbb68c468eda\") " pod="openshift-insights/insights-runtime-extractor-fncvg"
Oct 30 21:42:51.235760 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:51.235771    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f152360c-cf62-4d10-b2e5-b85ac7dfb739-metrics-tls\") pod \"dns-default-hbmxt\" (UID: \"f152360c-cf62-4d10-b2e5-b85ac7dfb739\") " pod="openshift-dns/dns-default-hbmxt"
Oct 30 21:42:51.238205 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:51.238183    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f152360c-cf62-4d10-b2e5-b85ac7dfb739-metrics-tls\") pod \"dns-default-hbmxt\" (UID: \"f152360c-cf62-4d10-b2e5-b85ac7dfb739\") " pod="openshift-dns/dns-default-hbmxt"
Oct 30 21:42:51.238304 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:51.238227    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/32a1c660-effa-4b6c-975a-dbb68c468eda-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fncvg\" (UID: \"32a1c660-effa-4b6c-975a-dbb68c468eda\") " pod="openshift-insights/insights-runtime-extractor-fncvg"
Oct 30 21:42:51.316503 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:51.316469    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-hbmxt"
Oct 30 21:42:51.317549 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:51.317529    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-fncvg"
Oct 30 21:42:51.690844 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:51.690806    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-hbmxt"]
Oct 30 21:42:51.731072 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:51.731045    4473 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf152360c_cf62_4d10_b2e5_b85ac7dfb739.slice/crio-4134d6b71ff1fdc42c80a5d9ca607c0e154f06ca50898d130a7bfb2594b11dd8 WatchSource:0}: Error finding container 4134d6b71ff1fdc42c80a5d9ca607c0e154f06ca50898d130a7bfb2594b11dd8: Status 404 returned error can't find the container with id 4134d6b71ff1fdc42c80a5d9ca607c0e154f06ca50898d130a7bfb2594b11dd8
Oct 30 21:42:51.745329 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:51.745298    4473 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32a1c660_effa_4b6c_975a_dbb68c468eda.slice/crio-28468281f0c9c11fd002f8ad7444439798ce5a66922e28384447be246c449cb6 WatchSource:0}: Error finding container 28468281f0c9c11fd002f8ad7444439798ce5a66922e28384447be246c449cb6: Status 404 returned error can't find the container with id 28468281f0c9c11fd002f8ad7444439798ce5a66922e28384447be246c449cb6
Oct 30 21:42:51.749224 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:51.749203    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-fncvg"]
Oct 30 21:42:52.437938 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:52.437884    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-hbmxt" event={"ID":"f152360c-cf62-4d10-b2e5-b85ac7dfb739","Type":"ContainerStarted","Data":"4134d6b71ff1fdc42c80a5d9ca607c0e154f06ca50898d130a7bfb2594b11dd8"}
Oct 30 21:42:52.439381 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:52.439362    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-fncvg" event={"ID":"32a1c660-effa-4b6c-975a-dbb68c468eda","Type":"ContainerStarted","Data":"a4c1e0ec8c87f2c3855d48a9b47a2ca26f5968f36466b104a05a9c3d6bb41348"}
Oct 30 21:42:52.439436 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:52.439405    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-fncvg" event={"ID":"32a1c660-effa-4b6c-975a-dbb68c468eda","Type":"ContainerStarted","Data":"28468281f0c9c11fd002f8ad7444439798ce5a66922e28384447be246c449cb6"}
Oct 30 21:42:53.445561 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:53.445521    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-fncvg" event={"ID":"32a1c660-effa-4b6c-975a-dbb68c468eda","Type":"ContainerStarted","Data":"0d52b25fc04fb48374972d4e0bf3d99c246e081342714af92b29df36774411bc"}
Oct 30 21:42:54.450981 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:54.450949    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-hbmxt" event={"ID":"f152360c-cf62-4d10-b2e5-b85ac7dfb739","Type":"ContainerStarted","Data":"a510aee4df6004e49605c78e37e3fd06dbaf9087c73b00c44b7b00e2201f8cf3"}
Oct 30 21:42:54.451444 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:54.451001    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-hbmxt" event={"ID":"f152360c-cf62-4d10-b2e5-b85ac7dfb739","Type":"ContainerStarted","Data":"dc4d91e5c1c11337eadfcb6d23c3e7c88b8226a53a5190af82022136f48ce202"}
Oct 30 21:42:54.451444 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:54.451083    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-hbmxt"
Oct 30 21:42:55.457084 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:55.457049    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-fncvg" event={"ID":"32a1c660-effa-4b6c-975a-dbb68c468eda","Type":"ContainerStarted","Data":"088714ed30fb6a059b537766990fecb984ca9c68765f8418a7ffdfd7c4efc96d"}
Oct 30 21:42:56.311891 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:56.311858    4473 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-57d5867d6f-94q6b"]
Oct 30 21:42:56.314661 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:56.314634    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-57d5867d6f-94q6b"
Oct 30 21:42:56.368280 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:56.368255    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Oct 30 21:42:56.374572 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:56.374546    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/60cba30b-6c5f-4409-a108-dc8f71232fd9-console-config\") pod \"console-57d5867d6f-94q6b\" (UID: \"60cba30b-6c5f-4409-a108-dc8f71232fd9\") " pod="openshift-console/console-57d5867d6f-94q6b"
Oct 30 21:42:56.374726 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:56.374587    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/60cba30b-6c5f-4409-a108-dc8f71232fd9-console-oauth-config\") pod \"console-57d5867d6f-94q6b\" (UID: \"60cba30b-6c5f-4409-a108-dc8f71232fd9\") " pod="openshift-console/console-57d5867d6f-94q6b"
Oct 30 21:42:56.374726 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:56.374616    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/60cba30b-6c5f-4409-a108-dc8f71232fd9-service-ca\") pod \"console-57d5867d6f-94q6b\" (UID: \"60cba30b-6c5f-4409-a108-dc8f71232fd9\") " pod="openshift-console/console-57d5867d6f-94q6b"
Oct 30 21:42:56.374838 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:56.374755    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/60cba30b-6c5f-4409-a108-dc8f71232fd9-oauth-serving-cert\") pod \"console-57d5867d6f-94q6b\" (UID: \"60cba30b-6c5f-4409-a108-dc8f71232fd9\") " pod="openshift-console/console-57d5867d6f-94q6b"
Oct 30 21:42:56.374838 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:56.374797    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/60cba30b-6c5f-4409-a108-dc8f71232fd9-tmp\") pod \"console-57d5867d6f-94q6b\" (UID: \"60cba30b-6c5f-4409-a108-dc8f71232fd9\") " pod="openshift-console/console-57d5867d6f-94q6b"
Oct 30 21:42:56.374838 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:56.374814    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jq66c\" (UniqueName: \"kubernetes.io/projected/60cba30b-6c5f-4409-a108-dc8f71232fd9-kube-api-access-jq66c\") pod \"console-57d5867d6f-94q6b\" (UID: \"60cba30b-6c5f-4409-a108-dc8f71232fd9\") " pod="openshift-console/console-57d5867d6f-94q6b"
Oct 30 21:42:56.374838 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:56.374835    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/60cba30b-6c5f-4409-a108-dc8f71232fd9-console-serving-cert\") pod \"console-57d5867d6f-94q6b\" (UID: \"60cba30b-6c5f-4409-a108-dc8f71232fd9\") " pod="openshift-console/console-57d5867d6f-94q6b"
Oct 30 21:42:56.419050 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:56.419016    4473 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Oct 30 21:42:56.419050 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:56.419021    4473 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Oct 30 21:42:56.419050 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:56.419025    4473 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Oct 30 21:42:56.419297 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:56.419052    4473 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Oct 30 21:42:56.419297 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:56.419088    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-n84t6\""
Oct 30 21:42:56.419297 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:56.419270    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Oct 30 21:42:56.419674 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:56.419657    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-57d5867d6f-94q6b"]
Oct 30 21:42:56.469599 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:56.469571    4473 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Oct 30 21:42:56.475882 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:56.475852    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/60cba30b-6c5f-4409-a108-dc8f71232fd9-oauth-serving-cert\") pod \"console-57d5867d6f-94q6b\" (UID: \"60cba30b-6c5f-4409-a108-dc8f71232fd9\") " pod="openshift-console/console-57d5867d6f-94q6b"
Oct 30 21:42:56.475974 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:56.475894    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/60cba30b-6c5f-4409-a108-dc8f71232fd9-tmp\") pod \"console-57d5867d6f-94q6b\" (UID: \"60cba30b-6c5f-4409-a108-dc8f71232fd9\") " pod="openshift-console/console-57d5867d6f-94q6b"
Oct 30 21:42:56.475974 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:56.475910    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jq66c\" (UniqueName: \"kubernetes.io/projected/60cba30b-6c5f-4409-a108-dc8f71232fd9-kube-api-access-jq66c\") pod \"console-57d5867d6f-94q6b\" (UID: \"60cba30b-6c5f-4409-a108-dc8f71232fd9\") " pod="openshift-console/console-57d5867d6f-94q6b"
Oct 30 21:42:56.475974 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:56.475952    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/60cba30b-6c5f-4409-a108-dc8f71232fd9-console-serving-cert\") pod \"console-57d5867d6f-94q6b\" (UID: \"60cba30b-6c5f-4409-a108-dc8f71232fd9\") " pod="openshift-console/console-57d5867d6f-94q6b"
Oct 30 21:42:56.476104 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:56.475978    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/60cba30b-6c5f-4409-a108-dc8f71232fd9-console-config\") pod \"console-57d5867d6f-94q6b\" (UID: \"60cba30b-6c5f-4409-a108-dc8f71232fd9\") " pod="openshift-console/console-57d5867d6f-94q6b"
Oct 30 21:42:56.476104 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:56.476003    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/60cba30b-6c5f-4409-a108-dc8f71232fd9-console-oauth-config\") pod \"console-57d5867d6f-94q6b\" (UID: \"60cba30b-6c5f-4409-a108-dc8f71232fd9\") " pod="openshift-console/console-57d5867d6f-94q6b"
Oct 30 21:42:56.476104 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:56.476020    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/60cba30b-6c5f-4409-a108-dc8f71232fd9-service-ca\") pod \"console-57d5867d6f-94q6b\" (UID: \"60cba30b-6c5f-4409-a108-dc8f71232fd9\") " pod="openshift-console/console-57d5867d6f-94q6b"
Oct 30 21:42:56.476383 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:56.476347    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/60cba30b-6c5f-4409-a108-dc8f71232fd9-tmp\") pod \"console-57d5867d6f-94q6b\" (UID: \"60cba30b-6c5f-4409-a108-dc8f71232fd9\") " pod="openshift-console/console-57d5867d6f-94q6b"
Oct 30 21:42:56.476713 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:56.476692    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/60cba30b-6c5f-4409-a108-dc8f71232fd9-console-config\") pod \"console-57d5867d6f-94q6b\" (UID: \"60cba30b-6c5f-4409-a108-dc8f71232fd9\") " pod="openshift-console/console-57d5867d6f-94q6b"
Oct 30 21:42:56.476805 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:56.476746    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/60cba30b-6c5f-4409-a108-dc8f71232fd9-service-ca\") pod \"console-57d5867d6f-94q6b\" (UID: \"60cba30b-6c5f-4409-a108-dc8f71232fd9\") " pod="openshift-console/console-57d5867d6f-94q6b"
Oct 30 21:42:56.476977 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:56.476958    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/60cba30b-6c5f-4409-a108-dc8f71232fd9-oauth-serving-cert\") pod \"console-57d5867d6f-94q6b\" (UID: \"60cba30b-6c5f-4409-a108-dc8f71232fd9\") " pod="openshift-console/console-57d5867d6f-94q6b"
Oct 30 21:42:56.478628 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:56.478607    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/60cba30b-6c5f-4409-a108-dc8f71232fd9-console-serving-cert\") pod \"console-57d5867d6f-94q6b\" (UID: \"60cba30b-6c5f-4409-a108-dc8f71232fd9\") " pod="openshift-console/console-57d5867d6f-94q6b"
Oct 30 21:42:56.478720 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:56.478639    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/60cba30b-6c5f-4409-a108-dc8f71232fd9-console-oauth-config\") pod \"console-57d5867d6f-94q6b\" (UID: \"60cba30b-6c5f-4409-a108-dc8f71232fd9\") " pod="openshift-console/console-57d5867d6f-94q6b"
Oct 30 21:42:56.533523 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:56.533490    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jq66c\" (UniqueName: \"kubernetes.io/projected/60cba30b-6c5f-4409-a108-dc8f71232fd9-kube-api-access-jq66c\") pod \"console-57d5867d6f-94q6b\" (UID: \"60cba30b-6c5f-4409-a108-dc8f71232fd9\") " pod="openshift-console/console-57d5867d6f-94q6b"
Oct 30 21:42:56.624649 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:56.624607    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-57d5867d6f-94q6b"
Oct 30 21:42:56.999357 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:42:56.999325    4473 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60cba30b_6c5f_4409_a108_dc8f71232fd9.slice/crio-d50fa3f5b0231adccdeeffba3b75731efb86bf04a8003c9f48a39b67763bf5ba WatchSource:0}: Error finding container d50fa3f5b0231adccdeeffba3b75731efb86bf04a8003c9f48a39b67763bf5ba: Status 404 returned error can't find the container with id d50fa3f5b0231adccdeeffba3b75731efb86bf04a8003c9f48a39b67763bf5ba
Oct 30 21:42:57.001493 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:57.001474    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-57d5867d6f-94q6b"]
Oct 30 21:42:57.464457 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:42:57.464423    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-57d5867d6f-94q6b" event={"ID":"60cba30b-6c5f-4409-a108-dc8f71232fd9","Type":"ContainerStarted","Data":"d50fa3f5b0231adccdeeffba3b75731efb86bf04a8003c9f48a39b67763bf5ba"}
Oct 30 21:43:00.477398 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:00.477316    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-57d5867d6f-94q6b" event={"ID":"60cba30b-6c5f-4409-a108-dc8f71232fd9","Type":"ContainerStarted","Data":"80db523a46a63b09be44c755ade961a23f8379f91908ccd28065c4dfad827768"}
Oct 30 21:43:01.553391 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:01.553363    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-krcxf"
Oct 30 21:43:04.459789 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:04.459756    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-hbmxt"
Oct 30 21:43:04.619386 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:04.619333    4473 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/console-57d5867d6f-94q6b" podStartSLOduration=5.467656184 podStartE2EDuration="8.619317974s" totalImagesPullingTime="3.15166179s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:42:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:43:00.59617675 +0000 UTC m=+26.369545234" watchObservedRunningTime="2026-10-30 21:43:04.619317974 +0000 UTC m=+30.392686453"
Oct 30 21:43:04.899828 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:04.899789    4473 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-kzr98"]
Oct 30 21:43:04.933532 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:04.933502    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-kzr98"
Oct 30 21:43:04.942140 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:04.942117    4473 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-5dc8d7d89b-b2pn2"]
Oct 30 21:43:04.955517 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:04.955493    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-b2pn2"
Oct 30 21:43:04.988828 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:04.988799    4473 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Oct 30 21:43:04.988828 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:04.988804    4473 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Oct 30 21:43:04.992185 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:04.992167    4473 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Oct 30 21:43:04.992247 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:04.992173    4473 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Oct 30 21:43:05.033010 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:05.032980    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a1c5b288-d0e3-4e50-a9eb-f2d2660b3718-kube-state-metrics-tls\") pod \"kube-state-metrics-5dc8d7d89b-b2pn2\" (UID: \"a1c5b288-d0e3-4e50-a9eb-f2d2660b3718\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-b2pn2"
Oct 30 21:43:05.033010 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:05.033015    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cv7j2\" (UniqueName: \"kubernetes.io/projected/a1c5b288-d0e3-4e50-a9eb-f2d2660b3718-kube-api-access-cv7j2\") pod \"kube-state-metrics-5dc8d7d89b-b2pn2\" (UID: \"a1c5b288-d0e3-4e50-a9eb-f2d2660b3718\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-b2pn2"
Oct 30 21:43:05.033187 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:05.033034    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a1c5b288-d0e3-4e50-a9eb-f2d2660b3718-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-5dc8d7d89b-b2pn2\" (UID: \"a1c5b288-d0e3-4e50-a9eb-f2d2660b3718\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-b2pn2"
Oct 30 21:43:05.033187 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:05.033055    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rwjwj\" (UniqueName: \"kubernetes.io/projected/a20fdb9f-75a2-4373-9337-72fa18ffef81-kube-api-access-rwjwj\") pod \"node-exporter-kzr98\" (UID: \"a20fdb9f-75a2-4373-9337-72fa18ffef81\") " pod="openshift-monitoring/node-exporter-kzr98"
Oct 30 21:43:05.033187 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:05.033157    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a20fdb9f-75a2-4373-9337-72fa18ffef81-node-exporter-accelerators-collector-config\") pod \"node-exporter-kzr98\" (UID: \"a20fdb9f-75a2-4373-9337-72fa18ffef81\") " pod="openshift-monitoring/node-exporter-kzr98"
Oct 30 21:43:05.033282 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:05.033187    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a20fdb9f-75a2-4373-9337-72fa18ffef81-root\") pod \"node-exporter-kzr98\" (UID: \"a20fdb9f-75a2-4373-9337-72fa18ffef81\") " pod="openshift-monitoring/node-exporter-kzr98"
Oct 30 21:43:05.033282 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:05.033210    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/a1c5b288-d0e3-4e50-a9eb-f2d2660b3718-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-5dc8d7d89b-b2pn2\" (UID: \"a1c5b288-d0e3-4e50-a9eb-f2d2660b3718\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-b2pn2"
Oct 30 21:43:05.033282 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:05.033254    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a20fdb9f-75a2-4373-9337-72fa18ffef81-node-exporter-textfile\") pod \"node-exporter-kzr98\" (UID: \"a20fdb9f-75a2-4373-9337-72fa18ffef81\") " pod="openshift-monitoring/node-exporter-kzr98"
Oct 30 21:43:05.033365 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:05.033304    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a20fdb9f-75a2-4373-9337-72fa18ffef81-sys\") pod \"node-exporter-kzr98\" (UID: \"a20fdb9f-75a2-4373-9337-72fa18ffef81\") " pod="openshift-monitoring/node-exporter-kzr98"
Oct 30 21:43:05.033405 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:05.033363    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a20fdb9f-75a2-4373-9337-72fa18ffef81-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-kzr98\" (UID: \"a20fdb9f-75a2-4373-9337-72fa18ffef81\") " pod="openshift-monitoring/node-exporter-kzr98"
Oct 30 21:43:05.033405 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:05.033381    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a20fdb9f-75a2-4373-9337-72fa18ffef81-node-exporter-tls\") pod \"node-exporter-kzr98\" (UID: \"a20fdb9f-75a2-4373-9337-72fa18ffef81\") " pod="openshift-monitoring/node-exporter-kzr98"
Oct 30 21:43:05.033405 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:05.033394    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/a1c5b288-d0e3-4e50-a9eb-f2d2660b3718-volume-directive-shadow\") pod \"kube-state-metrics-5dc8d7d89b-b2pn2\" (UID: \"a1c5b288-d0e3-4e50-a9eb-f2d2660b3718\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-b2pn2"
Oct 30 21:43:05.033492 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:05.033450    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a20fdb9f-75a2-4373-9337-72fa18ffef81-node-exporter-wtmp\") pod \"node-exporter-kzr98\" (UID: \"a20fdb9f-75a2-4373-9337-72fa18ffef81\") " pod="openshift-monitoring/node-exporter-kzr98"
Oct 30 21:43:05.033492 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:05.033472    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a20fdb9f-75a2-4373-9337-72fa18ffef81-metrics-client-ca\") pod \"node-exporter-kzr98\" (UID: \"a20fdb9f-75a2-4373-9337-72fa18ffef81\") " pod="openshift-monitoring/node-exporter-kzr98"
Oct 30 21:43:05.033492 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:05.033487    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a1c5b288-d0e3-4e50-a9eb-f2d2660b3718-metrics-client-ca\") pod \"kube-state-metrics-5dc8d7d89b-b2pn2\" (UID: \"a1c5b288-d0e3-4e50-a9eb-f2d2660b3718\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-b2pn2"
Oct 30 21:43:05.039392 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:05.039371    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Oct 30 21:43:05.039392 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:05.039391    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Oct 30 21:43:05.039392 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:05.039391    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Oct 30 21:43:05.039630 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:05.039468    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-cqvl7\""
Oct 30 21:43:05.039630 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:05.039529    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Oct 30 21:43:05.039630 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:05.039592    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-xc8xg\""
Oct 30 21:43:05.039933 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:05.039902    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-5dc8d7d89b-b2pn2"]
Oct 30 21:43:05.089159 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:05.089135    4473 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Oct 30 21:43:05.134579 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:05.134551    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a20fdb9f-75a2-4373-9337-72fa18ffef81-node-exporter-wtmp\") pod \"node-exporter-kzr98\" (UID: \"a20fdb9f-75a2-4373-9337-72fa18ffef81\") " pod="openshift-monitoring/node-exporter-kzr98"
Oct 30 21:43:05.134767 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:05.134585    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a20fdb9f-75a2-4373-9337-72fa18ffef81-metrics-client-ca\") pod \"node-exporter-kzr98\" (UID: \"a20fdb9f-75a2-4373-9337-72fa18ffef81\") " pod="openshift-monitoring/node-exporter-kzr98"
Oct 30 21:43:05.134767 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:05.134607    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a1c5b288-d0e3-4e50-a9eb-f2d2660b3718-metrics-client-ca\") pod \"kube-state-metrics-5dc8d7d89b-b2pn2\" (UID: \"a1c5b288-d0e3-4e50-a9eb-f2d2660b3718\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-b2pn2"
Oct 30 21:43:05.134767 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:05.134696    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a1c5b288-d0e3-4e50-a9eb-f2d2660b3718-kube-state-metrics-tls\") pod \"kube-state-metrics-5dc8d7d89b-b2pn2\" (UID: \"a1c5b288-d0e3-4e50-a9eb-f2d2660b3718\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-b2pn2"
Oct 30 21:43:05.134767 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:05.134743    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cv7j2\" (UniqueName: \"kubernetes.io/projected/a1c5b288-d0e3-4e50-a9eb-f2d2660b3718-kube-api-access-cv7j2\") pod \"kube-state-metrics-5dc8d7d89b-b2pn2\" (UID: \"a1c5b288-d0e3-4e50-a9eb-f2d2660b3718\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-b2pn2"
Oct 30 21:43:05.134767 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:05.134768    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a1c5b288-d0e3-4e50-a9eb-f2d2660b3718-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-5dc8d7d89b-b2pn2\" (UID: \"a1c5b288-d0e3-4e50-a9eb-f2d2660b3718\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-b2pn2"
Oct 30 21:43:05.135090 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:05.134789    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rwjwj\" (UniqueName: \"kubernetes.io/projected/a20fdb9f-75a2-4373-9337-72fa18ffef81-kube-api-access-rwjwj\") pod \"node-exporter-kzr98\" (UID: \"a20fdb9f-75a2-4373-9337-72fa18ffef81\") " pod="openshift-monitoring/node-exporter-kzr98"
Oct 30 21:43:05.135090 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:05.134816    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a20fdb9f-75a2-4373-9337-72fa18ffef81-node-exporter-accelerators-collector-config\") pod \"node-exporter-kzr98\" (UID: \"a20fdb9f-75a2-4373-9337-72fa18ffef81\") " pod="openshift-monitoring/node-exporter-kzr98"
Oct 30 21:43:05.135090 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:05.134840    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a20fdb9f-75a2-4373-9337-72fa18ffef81-root\") pod \"node-exporter-kzr98\" (UID: \"a20fdb9f-75a2-4373-9337-72fa18ffef81\") " pod="openshift-monitoring/node-exporter-kzr98"
Oct 30 21:43:05.135090 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:05.134875    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/a1c5b288-d0e3-4e50-a9eb-f2d2660b3718-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-5dc8d7d89b-b2pn2\" (UID: \"a1c5b288-d0e3-4e50-a9eb-f2d2660b3718\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-b2pn2"
Oct 30 21:43:05.135090 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:05.134905    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a20fdb9f-75a2-4373-9337-72fa18ffef81-node-exporter-textfile\") pod \"node-exporter-kzr98\" (UID: \"a20fdb9f-75a2-4373-9337-72fa18ffef81\") " pod="openshift-monitoring/node-exporter-kzr98"
Oct 30 21:43:05.135090 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:05.134957    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a20fdb9f-75a2-4373-9337-72fa18ffef81-sys\") pod \"node-exporter-kzr98\" (UID: \"a20fdb9f-75a2-4373-9337-72fa18ffef81\") " pod="openshift-monitoring/node-exporter-kzr98"
Oct 30 21:43:05.135090 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:05.134988    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a20fdb9f-75a2-4373-9337-72fa18ffef81-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-kzr98\" (UID: \"a20fdb9f-75a2-4373-9337-72fa18ffef81\") " pod="openshift-monitoring/node-exporter-kzr98"
Oct 30 21:43:05.135090 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:05.135016    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a20fdb9f-75a2-4373-9337-72fa18ffef81-node-exporter-tls\") pod \"node-exporter-kzr98\" (UID: \"a20fdb9f-75a2-4373-9337-72fa18ffef81\") " pod="openshift-monitoring/node-exporter-kzr98"
Oct 30 21:43:05.135090 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:05.135039    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/a1c5b288-d0e3-4e50-a9eb-f2d2660b3718-volume-directive-shadow\") pod \"kube-state-metrics-5dc8d7d89b-b2pn2\" (UID: \"a1c5b288-d0e3-4e50-a9eb-f2d2660b3718\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-b2pn2"
Oct 30 21:43:05.135531 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:05.135359    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a20fdb9f-75a2-4373-9337-72fa18ffef81-metrics-client-ca\") pod \"node-exporter-kzr98\" (UID: \"a20fdb9f-75a2-4373-9337-72fa18ffef81\") " pod="openshift-monitoring/node-exporter-kzr98"
Oct 30 21:43:05.135531 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:05.134742    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a20fdb9f-75a2-4373-9337-72fa18ffef81-node-exporter-wtmp\") pod \"node-exporter-kzr98\" (UID: \"a20fdb9f-75a2-4373-9337-72fa18ffef81\") " pod="openshift-monitoring/node-exporter-kzr98"
Oct 30 21:43:05.135531 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:05.135395    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a20fdb9f-75a2-4373-9337-72fa18ffef81-node-exporter-textfile\") pod \"node-exporter-kzr98\" (UID: \"a20fdb9f-75a2-4373-9337-72fa18ffef81\") " pod="openshift-monitoring/node-exporter-kzr98"
Oct 30 21:43:05.135531 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:05.135396    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a20fdb9f-75a2-4373-9337-72fa18ffef81-node-exporter-accelerators-collector-config\") pod \"node-exporter-kzr98\" (UID: \"a20fdb9f-75a2-4373-9337-72fa18ffef81\") " pod="openshift-monitoring/node-exporter-kzr98"
Oct 30 21:43:05.135531 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:05.135440    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/a1c5b288-d0e3-4e50-a9eb-f2d2660b3718-volume-directive-shadow\") pod \"kube-state-metrics-5dc8d7d89b-b2pn2\" (UID: \"a1c5b288-d0e3-4e50-a9eb-f2d2660b3718\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-b2pn2"
Oct 30 21:43:05.135531 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:05.135454    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a20fdb9f-75a2-4373-9337-72fa18ffef81-sys\") pod \"node-exporter-kzr98\" (UID: \"a20fdb9f-75a2-4373-9337-72fa18ffef81\") " pod="openshift-monitoring/node-exporter-kzr98"
Oct 30 21:43:05.135531 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:05.135447    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a1c5b288-d0e3-4e50-a9eb-f2d2660b3718-metrics-client-ca\") pod \"kube-state-metrics-5dc8d7d89b-b2pn2\" (UID: \"a1c5b288-d0e3-4e50-a9eb-f2d2660b3718\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-b2pn2"
Oct 30 21:43:05.135531 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:05.135483    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a20fdb9f-75a2-4373-9337-72fa18ffef81-root\") pod \"node-exporter-kzr98\" (UID: \"a20fdb9f-75a2-4373-9337-72fa18ffef81\") " pod="openshift-monitoring/node-exporter-kzr98"
Oct 30 21:43:05.136277 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:05.136254    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/a1c5b288-d0e3-4e50-a9eb-f2d2660b3718-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-5dc8d7d89b-b2pn2\" (UID: \"a1c5b288-d0e3-4e50-a9eb-f2d2660b3718\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-b2pn2"
Oct 30 21:43:05.137651 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:05.137630    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a20fdb9f-75a2-4373-9337-72fa18ffef81-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-kzr98\" (UID: \"a20fdb9f-75a2-4373-9337-72fa18ffef81\") " pod="openshift-monitoring/node-exporter-kzr98"
Oct 30 21:43:05.137737 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:05.137677    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a1c5b288-d0e3-4e50-a9eb-f2d2660b3718-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-5dc8d7d89b-b2pn2\" (UID: \"a1c5b288-d0e3-4e50-a9eb-f2d2660b3718\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-b2pn2"
Oct 30 21:43:05.137907 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:05.137891    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a1c5b288-d0e3-4e50-a9eb-f2d2660b3718-kube-state-metrics-tls\") pod \"kube-state-metrics-5dc8d7d89b-b2pn2\" (UID: \"a1c5b288-d0e3-4e50-a9eb-f2d2660b3718\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-b2pn2"
Oct 30 21:43:05.137989 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:05.137939    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a20fdb9f-75a2-4373-9337-72fa18ffef81-node-exporter-tls\") pod \"node-exporter-kzr98\" (UID: \"a20fdb9f-75a2-4373-9337-72fa18ffef81\") " pod="openshift-monitoring/node-exporter-kzr98"
Oct 30 21:43:05.192521 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:05.192455    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cv7j2\" (UniqueName: \"kubernetes.io/projected/a1c5b288-d0e3-4e50-a9eb-f2d2660b3718-kube-api-access-cv7j2\") pod \"kube-state-metrics-5dc8d7d89b-b2pn2\" (UID: \"a1c5b288-d0e3-4e50-a9eb-f2d2660b3718\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-b2pn2"
Oct 30 21:43:05.192764 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:05.192743    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rwjwj\" (UniqueName: \"kubernetes.io/projected/a20fdb9f-75a2-4373-9337-72fa18ffef81-kube-api-access-rwjwj\") pod \"node-exporter-kzr98\" (UID: \"a20fdb9f-75a2-4373-9337-72fa18ffef81\") " pod="openshift-monitoring/node-exporter-kzr98"
Oct 30 21:43:05.284180 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:05.284147    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-kzr98"
Oct 30 21:43:05.324102 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:43:05.324070    4473 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda20fdb9f_75a2_4373_9337_72fa18ffef81.slice/crio-a04c4cb110db7e93a56d32bde7011bdc9a9a349f19ecbee8f5886d7e8bd29d03 WatchSource:0}: Error finding container a04c4cb110db7e93a56d32bde7011bdc9a9a349f19ecbee8f5886d7e8bd29d03: Status 404 returned error can't find the container with id a04c4cb110db7e93a56d32bde7011bdc9a9a349f19ecbee8f5886d7e8bd29d03
Oct 30 21:43:05.326385 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:05.326361    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-b2pn2"
Oct 30 21:43:05.493973 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:05.493867    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-kzr98" event={"ID":"a20fdb9f-75a2-4373-9337-72fa18ffef81","Type":"ContainerStarted","Data":"a04c4cb110db7e93a56d32bde7011bdc9a9a349f19ecbee8f5886d7e8bd29d03"}
Oct 30 21:43:05.624027 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:05.623993    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-9st74"
Oct 30 21:43:05.757754 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:05.757678    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-5dc8d7d89b-b2pn2"]
Oct 30 21:43:05.760363 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:43:05.760333    4473 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1c5b288_d0e3_4e50_a9eb_f2d2660b3718.slice/crio-a9a1af315e83e51c3d9416f319aed238e2cd3448c63be3dcd5936c0d72c6de45 WatchSource:0}: Error finding container a9a1af315e83e51c3d9416f319aed238e2cd3448c63be3dcd5936c0d72c6de45: Status 404 returned error can't find the container with id a9a1af315e83e51c3d9416f319aed238e2cd3448c63be3dcd5936c0d72c6de45
Oct 30 21:43:06.499377 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:06.499333    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-b2pn2" event={"ID":"a1c5b288-d0e3-4e50-a9eb-f2d2660b3718","Type":"ContainerStarted","Data":"a9a1af315e83e51c3d9416f319aed238e2cd3448c63be3dcd5936c0d72c6de45"}
Oct 30 21:43:06.625394 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:06.625359    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-57d5867d6f-94q6b"
Oct 30 21:43:06.625575 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:06.625439    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-57d5867d6f-94q6b"
Oct 30 21:43:06.630290 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:06.630269    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-57d5867d6f-94q6b"
Oct 30 21:43:07.351551 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:07.351521    4473 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Oct 30 21:43:07.365935 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:07.365897    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:43:07.411703 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:07.411605    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Oct 30 21:43:07.418839 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:07.418808    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Oct 30 21:43:07.456438 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:07.456402    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c570a4d1-5a01-4beb-a659-1ac75ea3e768-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"c570a4d1-5a01-4beb-a659-1ac75ea3e768\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:43:07.456602 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:07.456451    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/c570a4d1-5a01-4beb-a659-1ac75ea3e768-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"c570a4d1-5a01-4beb-a659-1ac75ea3e768\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:43:07.456602 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:07.456477    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c570a4d1-5a01-4beb-a659-1ac75ea3e768-web-config\") pod \"alertmanager-main-0\" (UID: \"c570a4d1-5a01-4beb-a659-1ac75ea3e768\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:43:07.456602 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:07.456572    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c570a4d1-5a01-4beb-a659-1ac75ea3e768-config-out\") pod \"alertmanager-main-0\" (UID: \"c570a4d1-5a01-4beb-a659-1ac75ea3e768\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:43:07.456602 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:07.456599    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bz72j\" (UniqueName: \"kubernetes.io/projected/c570a4d1-5a01-4beb-a659-1ac75ea3e768-kube-api-access-bz72j\") pod \"alertmanager-main-0\" (UID: \"c570a4d1-5a01-4beb-a659-1ac75ea3e768\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:43:07.456756 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:07.456620    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c570a4d1-5a01-4beb-a659-1ac75ea3e768-tls-assets\") pod \"alertmanager-main-0\" (UID: \"c570a4d1-5a01-4beb-a659-1ac75ea3e768\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:43:07.456756 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:07.456639    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c570a4d1-5a01-4beb-a659-1ac75ea3e768-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"c570a4d1-5a01-4beb-a659-1ac75ea3e768\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:43:07.456756 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:07.456665    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c570a4d1-5a01-4beb-a659-1ac75ea3e768-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"c570a4d1-5a01-4beb-a659-1ac75ea3e768\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:43:07.456756 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:07.456682    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/c570a4d1-5a01-4beb-a659-1ac75ea3e768-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"c570a4d1-5a01-4beb-a659-1ac75ea3e768\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:43:07.456756 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:07.456704    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/c570a4d1-5a01-4beb-a659-1ac75ea3e768-config-volume\") pod \"alertmanager-main-0\" (UID: \"c570a4d1-5a01-4beb-a659-1ac75ea3e768\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:43:07.456756 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:07.456730    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/c570a4d1-5a01-4beb-a659-1ac75ea3e768-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"c570a4d1-5a01-4beb-a659-1ac75ea3e768\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:43:07.456756 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:07.456750    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c570a4d1-5a01-4beb-a659-1ac75ea3e768-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"c570a4d1-5a01-4beb-a659-1ac75ea3e768\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:43:07.457063 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:07.456783    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/c570a4d1-5a01-4beb-a659-1ac75ea3e768-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"c570a4d1-5a01-4beb-a659-1ac75ea3e768\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:43:07.461346 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:07.461235    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Oct 30 21:43:07.461346 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:07.461247    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Oct 30 21:43:07.461346 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:07.461239    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Oct 30 21:43:07.461346 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:07.461311    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Oct 30 21:43:07.461346 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:07.461331    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-xpr7b\""
Oct 30 21:43:07.461667 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:07.461342    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Oct 30 21:43:07.461667 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:07.461395    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Oct 30 21:43:07.461667 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:07.461460    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Oct 30 21:43:07.504498 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:07.504468    4473 generic.go:356] "Generic (PLEG): container finished" podID="a20fdb9f-75a2-4373-9337-72fa18ffef81" containerID="0274efa3c86638973867c57a79c12ed5ea00fef8f12f4c72bc07e90d65d1e274" exitCode=0
Oct 30 21:43:07.504893 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:07.504551    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-kzr98" event={"ID":"a20fdb9f-75a2-4373-9337-72fa18ffef81","Type":"ContainerDied","Data":"0274efa3c86638973867c57a79c12ed5ea00fef8f12f4c72bc07e90d65d1e274"}
Oct 30 21:43:07.509376 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:07.509356    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-57d5867d6f-94q6b"
Oct 30 21:43:07.520332 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:07.520308    4473 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Oct 30 21:43:07.557784 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:07.557752    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c570a4d1-5a01-4beb-a659-1ac75ea3e768-config-out\") pod \"alertmanager-main-0\" (UID: \"c570a4d1-5a01-4beb-a659-1ac75ea3e768\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:43:07.557951 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:07.557796    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bz72j\" (UniqueName: \"kubernetes.io/projected/c570a4d1-5a01-4beb-a659-1ac75ea3e768-kube-api-access-bz72j\") pod \"alertmanager-main-0\" (UID: \"c570a4d1-5a01-4beb-a659-1ac75ea3e768\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:43:07.557951 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:07.557828    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c570a4d1-5a01-4beb-a659-1ac75ea3e768-tls-assets\") pod \"alertmanager-main-0\" (UID: \"c570a4d1-5a01-4beb-a659-1ac75ea3e768\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:43:07.557951 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:07.557876    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c570a4d1-5a01-4beb-a659-1ac75ea3e768-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"c570a4d1-5a01-4beb-a659-1ac75ea3e768\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:43:07.558130 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:07.557965    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c570a4d1-5a01-4beb-a659-1ac75ea3e768-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"c570a4d1-5a01-4beb-a659-1ac75ea3e768\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:43:07.558130 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:07.557989    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/c570a4d1-5a01-4beb-a659-1ac75ea3e768-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"c570a4d1-5a01-4beb-a659-1ac75ea3e768\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:43:07.558130 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:07.558017    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/c570a4d1-5a01-4beb-a659-1ac75ea3e768-config-volume\") pod \"alertmanager-main-0\" (UID: \"c570a4d1-5a01-4beb-a659-1ac75ea3e768\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:43:07.558130 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:07.558045    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/c570a4d1-5a01-4beb-a659-1ac75ea3e768-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"c570a4d1-5a01-4beb-a659-1ac75ea3e768\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:43:07.558130 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:07.558071    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c570a4d1-5a01-4beb-a659-1ac75ea3e768-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"c570a4d1-5a01-4beb-a659-1ac75ea3e768\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:43:07.558130 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:07.558102    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/c570a4d1-5a01-4beb-a659-1ac75ea3e768-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"c570a4d1-5a01-4beb-a659-1ac75ea3e768\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:43:07.558431 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:07.558190    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c570a4d1-5a01-4beb-a659-1ac75ea3e768-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"c570a4d1-5a01-4beb-a659-1ac75ea3e768\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:43:07.558431 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:07.558319    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/c570a4d1-5a01-4beb-a659-1ac75ea3e768-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"c570a4d1-5a01-4beb-a659-1ac75ea3e768\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:43:07.558431 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:07.558348    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c570a4d1-5a01-4beb-a659-1ac75ea3e768-web-config\") pod \"alertmanager-main-0\" (UID: \"c570a4d1-5a01-4beb-a659-1ac75ea3e768\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:43:07.558937 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:07.558852    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c570a4d1-5a01-4beb-a659-1ac75ea3e768-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"c570a4d1-5a01-4beb-a659-1ac75ea3e768\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:43:07.562463 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:07.561206    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c570a4d1-5a01-4beb-a659-1ac75ea3e768-tls-assets\") pod \"alertmanager-main-0\" (UID: \"c570a4d1-5a01-4beb-a659-1ac75ea3e768\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:43:07.562463 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:07.561573    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c570a4d1-5a01-4beb-a659-1ac75ea3e768-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"c570a4d1-5a01-4beb-a659-1ac75ea3e768\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:43:07.562463 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:07.561734    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/c570a4d1-5a01-4beb-a659-1ac75ea3e768-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"c570a4d1-5a01-4beb-a659-1ac75ea3e768\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:43:07.562463 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:07.562124    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/c570a4d1-5a01-4beb-a659-1ac75ea3e768-config-volume\") pod \"alertmanager-main-0\" (UID: \"c570a4d1-5a01-4beb-a659-1ac75ea3e768\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:43:07.562463 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:07.562165    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/c570a4d1-5a01-4beb-a659-1ac75ea3e768-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"c570a4d1-5a01-4beb-a659-1ac75ea3e768\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:43:07.562463 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:07.562352    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c570a4d1-5a01-4beb-a659-1ac75ea3e768-config-out\") pod \"alertmanager-main-0\" (UID: \"c570a4d1-5a01-4beb-a659-1ac75ea3e768\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:43:07.563023 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:07.562719    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c570a4d1-5a01-4beb-a659-1ac75ea3e768-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"c570a4d1-5a01-4beb-a659-1ac75ea3e768\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:43:07.563023 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:07.562735    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c570a4d1-5a01-4beb-a659-1ac75ea3e768-web-config\") pod \"alertmanager-main-0\" (UID: \"c570a4d1-5a01-4beb-a659-1ac75ea3e768\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:43:07.564358 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:07.564334    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/c570a4d1-5a01-4beb-a659-1ac75ea3e768-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"c570a4d1-5a01-4beb-a659-1ac75ea3e768\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:43:07.564723 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:07.564702    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/c570a4d1-5a01-4beb-a659-1ac75ea3e768-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"c570a4d1-5a01-4beb-a659-1ac75ea3e768\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:43:07.565468 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:07.565444    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c570a4d1-5a01-4beb-a659-1ac75ea3e768-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"c570a4d1-5a01-4beb-a659-1ac75ea3e768\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:43:07.584460 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:07.584435    4473 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-68886bc5bd-pbpz9"]
Oct 30 21:43:07.594459 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:07.594434    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-68886bc5bd-pbpz9"
Oct 30 21:43:07.618021 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:07.617993    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bz72j\" (UniqueName: \"kubernetes.io/projected/c570a4d1-5a01-4beb-a659-1ac75ea3e768-kube-api-access-bz72j\") pod \"alertmanager-main-0\" (UID: \"c570a4d1-5a01-4beb-a659-1ac75ea3e768\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:43:07.652740 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:07.652712    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Oct 30 21:43:07.652740 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:07.652734    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Oct 30 21:43:07.652950 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:07.652720    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Oct 30 21:43:07.652950 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:07.652724    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Oct 30 21:43:07.652950 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:07.652786    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Oct 30 21:43:07.652950 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:07.652850    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-51dk0o77b7jlf\""
Oct 30 21:43:07.652950 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:07.652896    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-sh9t2\""
Oct 30 21:43:07.653187 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:07.653171    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-68886bc5bd-pbpz9"]
Oct 30 21:43:07.658857 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:07.658835    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/dd17e003-7b46-4f0d-b4e6-fad470cb81e8-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-68886bc5bd-pbpz9\" (UID: \"dd17e003-7b46-4f0d-b4e6-fad470cb81e8\") " pod="openshift-monitoring/thanos-querier-68886bc5bd-pbpz9"
Oct 30 21:43:07.658975 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:07.658870    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/dd17e003-7b46-4f0d-b4e6-fad470cb81e8-tmp\") pod \"thanos-querier-68886bc5bd-pbpz9\" (UID: \"dd17e003-7b46-4f0d-b4e6-fad470cb81e8\") " pod="openshift-monitoring/thanos-querier-68886bc5bd-pbpz9"
Oct 30 21:43:07.658975 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:07.658899    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/dd17e003-7b46-4f0d-b4e6-fad470cb81e8-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-68886bc5bd-pbpz9\" (UID: \"dd17e003-7b46-4f0d-b4e6-fad470cb81e8\") " pod="openshift-monitoring/thanos-querier-68886bc5bd-pbpz9"
Oct 30 21:43:07.659102 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:07.659077    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/dd17e003-7b46-4f0d-b4e6-fad470cb81e8-secret-thanos-querier-tls\") pod \"thanos-querier-68886bc5bd-pbpz9\" (UID: \"dd17e003-7b46-4f0d-b4e6-fad470cb81e8\") " pod="openshift-monitoring/thanos-querier-68886bc5bd-pbpz9"
Oct 30 21:43:07.659136 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:07.659121    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b7wxq\" (UniqueName: \"kubernetes.io/projected/dd17e003-7b46-4f0d-b4e6-fad470cb81e8-kube-api-access-b7wxq\") pod \"thanos-querier-68886bc5bd-pbpz9\" (UID: \"dd17e003-7b46-4f0d-b4e6-fad470cb81e8\") " pod="openshift-monitoring/thanos-querier-68886bc5bd-pbpz9"
Oct 30 21:43:07.659181 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:07.659166    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/dd17e003-7b46-4f0d-b4e6-fad470cb81e8-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-68886bc5bd-pbpz9\" (UID: \"dd17e003-7b46-4f0d-b4e6-fad470cb81e8\") " pod="openshift-monitoring/thanos-querier-68886bc5bd-pbpz9"
Oct 30 21:43:07.659223 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:07.659209    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/dd17e003-7b46-4f0d-b4e6-fad470cb81e8-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-68886bc5bd-pbpz9\" (UID: \"dd17e003-7b46-4f0d-b4e6-fad470cb81e8\") " pod="openshift-monitoring/thanos-querier-68886bc5bd-pbpz9"
Oct 30 21:43:07.659263 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:07.659250    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dd17e003-7b46-4f0d-b4e6-fad470cb81e8-metrics-client-ca\") pod \"thanos-querier-68886bc5bd-pbpz9\" (UID: \"dd17e003-7b46-4f0d-b4e6-fad470cb81e8\") " pod="openshift-monitoring/thanos-querier-68886bc5bd-pbpz9"
Oct 30 21:43:07.659298 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:07.659277    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/dd17e003-7b46-4f0d-b4e6-fad470cb81e8-secret-grpc-tls\") pod \"thanos-querier-68886bc5bd-pbpz9\" (UID: \"dd17e003-7b46-4f0d-b4e6-fad470cb81e8\") " pod="openshift-monitoring/thanos-querier-68886bc5bd-pbpz9"
Oct 30 21:43:07.760031 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:07.760001    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/dd17e003-7b46-4f0d-b4e6-fad470cb81e8-secret-thanos-querier-tls\") pod \"thanos-querier-68886bc5bd-pbpz9\" (UID: \"dd17e003-7b46-4f0d-b4e6-fad470cb81e8\") " pod="openshift-monitoring/thanos-querier-68886bc5bd-pbpz9"
Oct 30 21:43:07.760200 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:07.760035    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b7wxq\" (UniqueName: \"kubernetes.io/projected/dd17e003-7b46-4f0d-b4e6-fad470cb81e8-kube-api-access-b7wxq\") pod \"thanos-querier-68886bc5bd-pbpz9\" (UID: \"dd17e003-7b46-4f0d-b4e6-fad470cb81e8\") " pod="openshift-monitoring/thanos-querier-68886bc5bd-pbpz9"
Oct 30 21:43:07.760200 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:07.760087    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/dd17e003-7b46-4f0d-b4e6-fad470cb81e8-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-68886bc5bd-pbpz9\" (UID: \"dd17e003-7b46-4f0d-b4e6-fad470cb81e8\") " pod="openshift-monitoring/thanos-querier-68886bc5bd-pbpz9"
Oct 30 21:43:07.760200 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:07.760123    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/dd17e003-7b46-4f0d-b4e6-fad470cb81e8-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-68886bc5bd-pbpz9\" (UID: \"dd17e003-7b46-4f0d-b4e6-fad470cb81e8\") " pod="openshift-monitoring/thanos-querier-68886bc5bd-pbpz9"
Oct 30 21:43:07.760200 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:07.760156    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dd17e003-7b46-4f0d-b4e6-fad470cb81e8-metrics-client-ca\") pod \"thanos-querier-68886bc5bd-pbpz9\" (UID: \"dd17e003-7b46-4f0d-b4e6-fad470cb81e8\") " pod="openshift-monitoring/thanos-querier-68886bc5bd-pbpz9"
Oct 30 21:43:07.760200 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:07.760183    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/dd17e003-7b46-4f0d-b4e6-fad470cb81e8-secret-grpc-tls\") pod \"thanos-querier-68886bc5bd-pbpz9\" (UID: \"dd17e003-7b46-4f0d-b4e6-fad470cb81e8\") " pod="openshift-monitoring/thanos-querier-68886bc5bd-pbpz9"
Oct 30 21:43:07.760451 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:07.760225    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/dd17e003-7b46-4f0d-b4e6-fad470cb81e8-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-68886bc5bd-pbpz9\" (UID: \"dd17e003-7b46-4f0d-b4e6-fad470cb81e8\") " pod="openshift-monitoring/thanos-querier-68886bc5bd-pbpz9"
Oct 30 21:43:07.760451 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:07.760257    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/dd17e003-7b46-4f0d-b4e6-fad470cb81e8-tmp\") pod \"thanos-querier-68886bc5bd-pbpz9\" (UID: \"dd17e003-7b46-4f0d-b4e6-fad470cb81e8\") " pod="openshift-monitoring/thanos-querier-68886bc5bd-pbpz9"
Oct 30 21:43:07.760451 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:07.760288    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/dd17e003-7b46-4f0d-b4e6-fad470cb81e8-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-68886bc5bd-pbpz9\" (UID: \"dd17e003-7b46-4f0d-b4e6-fad470cb81e8\") " pod="openshift-monitoring/thanos-querier-68886bc5bd-pbpz9"
Oct 30 21:43:07.760809 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:07.760779    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/dd17e003-7b46-4f0d-b4e6-fad470cb81e8-tmp\") pod \"thanos-querier-68886bc5bd-pbpz9\" (UID: \"dd17e003-7b46-4f0d-b4e6-fad470cb81e8\") " pod="openshift-monitoring/thanos-querier-68886bc5bd-pbpz9"
Oct 30 21:43:07.761193 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:07.761117    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dd17e003-7b46-4f0d-b4e6-fad470cb81e8-metrics-client-ca\") pod \"thanos-querier-68886bc5bd-pbpz9\" (UID: \"dd17e003-7b46-4f0d-b4e6-fad470cb81e8\") " pod="openshift-monitoring/thanos-querier-68886bc5bd-pbpz9"
Oct 30 21:43:07.763123 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:07.763097    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/dd17e003-7b46-4f0d-b4e6-fad470cb81e8-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-68886bc5bd-pbpz9\" (UID: \"dd17e003-7b46-4f0d-b4e6-fad470cb81e8\") " pod="openshift-monitoring/thanos-querier-68886bc5bd-pbpz9"
Oct 30 21:43:07.763226 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:07.763161    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/dd17e003-7b46-4f0d-b4e6-fad470cb81e8-secret-grpc-tls\") pod \"thanos-querier-68886bc5bd-pbpz9\" (UID: \"dd17e003-7b46-4f0d-b4e6-fad470cb81e8\") " pod="openshift-monitoring/thanos-querier-68886bc5bd-pbpz9"
Oct 30 21:43:07.763339 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:07.763320    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/dd17e003-7b46-4f0d-b4e6-fad470cb81e8-secret-thanos-querier-tls\") pod \"thanos-querier-68886bc5bd-pbpz9\" (UID: \"dd17e003-7b46-4f0d-b4e6-fad470cb81e8\") " pod="openshift-monitoring/thanos-querier-68886bc5bd-pbpz9"
Oct 30 21:43:07.763451 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:07.763428    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/dd17e003-7b46-4f0d-b4e6-fad470cb81e8-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-68886bc5bd-pbpz9\" (UID: \"dd17e003-7b46-4f0d-b4e6-fad470cb81e8\") " pod="openshift-monitoring/thanos-querier-68886bc5bd-pbpz9"
Oct 30 21:43:07.763486 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:07.763469    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/dd17e003-7b46-4f0d-b4e6-fad470cb81e8-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-68886bc5bd-pbpz9\" (UID: \"dd17e003-7b46-4f0d-b4e6-fad470cb81e8\") " pod="openshift-monitoring/thanos-querier-68886bc5bd-pbpz9"
Oct 30 21:43:07.763854 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:07.763839    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/dd17e003-7b46-4f0d-b4e6-fad470cb81e8-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-68886bc5bd-pbpz9\" (UID: \"dd17e003-7b46-4f0d-b4e6-fad470cb81e8\") " pod="openshift-monitoring/thanos-querier-68886bc5bd-pbpz9"
Oct 30 21:43:07.778403 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:07.778346    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:43:07.817985 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:07.817960    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b7wxq\" (UniqueName: \"kubernetes.io/projected/dd17e003-7b46-4f0d-b4e6-fad470cb81e8-kube-api-access-b7wxq\") pod \"thanos-querier-68886bc5bd-pbpz9\" (UID: \"dd17e003-7b46-4f0d-b4e6-fad470cb81e8\") " pod="openshift-monitoring/thanos-querier-68886bc5bd-pbpz9"
Oct 30 21:43:07.937192 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:07.937161    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-68886bc5bd-pbpz9"
Oct 30 21:43:07.963503 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:07.963474    4473 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-6dbcc9bbc4-gjtwp"]
Oct 30 21:43:07.995702 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:07.995667    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6dbcc9bbc4-gjtwp"
Oct 30 21:43:08.029141 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:08.029015    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6dbcc9bbc4-gjtwp"]
Oct 30 21:43:08.054254 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:08.054228    4473 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Oct 30 21:43:08.063592 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:08.063512    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4460b81d-e403-41f8-a83a-2067c6f51086-service-ca\") pod \"console-6dbcc9bbc4-gjtwp\" (UID: \"4460b81d-e403-41f8-a83a-2067c6f51086\") " pod="openshift-console/console-6dbcc9bbc4-gjtwp"
Oct 30 21:43:08.063743 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:08.063666    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4460b81d-e403-41f8-a83a-2067c6f51086-oauth-serving-cert\") pod \"console-6dbcc9bbc4-gjtwp\" (UID: \"4460b81d-e403-41f8-a83a-2067c6f51086\") " pod="openshift-console/console-6dbcc9bbc4-gjtwp"
Oct 30 21:43:08.063743 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:08.063710    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4460b81d-e403-41f8-a83a-2067c6f51086-console-config\") pod \"console-6dbcc9bbc4-gjtwp\" (UID: \"4460b81d-e403-41f8-a83a-2067c6f51086\") " pod="openshift-console/console-6dbcc9bbc4-gjtwp"
Oct 30 21:43:08.063827 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:08.063792    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4460b81d-e403-41f8-a83a-2067c6f51086-console-serving-cert\") pod \"console-6dbcc9bbc4-gjtwp\" (UID: \"4460b81d-e403-41f8-a83a-2067c6f51086\") " pod="openshift-console/console-6dbcc9bbc4-gjtwp"
Oct 30 21:43:08.063869 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:08.063830    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4460b81d-e403-41f8-a83a-2067c6f51086-console-oauth-config\") pod \"console-6dbcc9bbc4-gjtwp\" (UID: \"4460b81d-e403-41f8-a83a-2067c6f51086\") " pod="openshift-console/console-6dbcc9bbc4-gjtwp"
Oct 30 21:43:08.063947 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:08.063898    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4460b81d-e403-41f8-a83a-2067c6f51086-tmp\") pod \"console-6dbcc9bbc4-gjtwp\" (UID: \"4460b81d-e403-41f8-a83a-2067c6f51086\") " pod="openshift-console/console-6dbcc9bbc4-gjtwp"
Oct 30 21:43:08.063990 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:08.063913    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4460b81d-e403-41f8-a83a-2067c6f51086-trusted-ca-bundle\") pod \"console-6dbcc9bbc4-gjtwp\" (UID: \"4460b81d-e403-41f8-a83a-2067c6f51086\") " pod="openshift-console/console-6dbcc9bbc4-gjtwp"
Oct 30 21:43:08.064030 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:08.063991    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c4xx8\" (UniqueName: \"kubernetes.io/projected/4460b81d-e403-41f8-a83a-2067c6f51086-kube-api-access-c4xx8\") pod \"console-6dbcc9bbc4-gjtwp\" (UID: \"4460b81d-e403-41f8-a83a-2067c6f51086\") " pod="openshift-console/console-6dbcc9bbc4-gjtwp"
Oct 30 21:43:08.164309 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:08.164285    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4460b81d-e403-41f8-a83a-2067c6f51086-oauth-serving-cert\") pod \"console-6dbcc9bbc4-gjtwp\" (UID: \"4460b81d-e403-41f8-a83a-2067c6f51086\") " pod="openshift-console/console-6dbcc9bbc4-gjtwp"
Oct 30 21:43:08.164370 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:08.164314    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4460b81d-e403-41f8-a83a-2067c6f51086-console-config\") pod \"console-6dbcc9bbc4-gjtwp\" (UID: \"4460b81d-e403-41f8-a83a-2067c6f51086\") " pod="openshift-console/console-6dbcc9bbc4-gjtwp"
Oct 30 21:43:08.164370 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:08.164340    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4460b81d-e403-41f8-a83a-2067c6f51086-console-serving-cert\") pod \"console-6dbcc9bbc4-gjtwp\" (UID: \"4460b81d-e403-41f8-a83a-2067c6f51086\") " pod="openshift-console/console-6dbcc9bbc4-gjtwp"
Oct 30 21:43:08.164370 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:08.164366    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4460b81d-e403-41f8-a83a-2067c6f51086-console-oauth-config\") pod \"console-6dbcc9bbc4-gjtwp\" (UID: \"4460b81d-e403-41f8-a83a-2067c6f51086\") " pod="openshift-console/console-6dbcc9bbc4-gjtwp"
Oct 30 21:43:08.164579 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:08.164394    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4460b81d-e403-41f8-a83a-2067c6f51086-tmp\") pod \"console-6dbcc9bbc4-gjtwp\" (UID: \"4460b81d-e403-41f8-a83a-2067c6f51086\") " pod="openshift-console/console-6dbcc9bbc4-gjtwp"
Oct 30 21:43:08.164579 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:08.164419    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4460b81d-e403-41f8-a83a-2067c6f51086-trusted-ca-bundle\") pod \"console-6dbcc9bbc4-gjtwp\" (UID: \"4460b81d-e403-41f8-a83a-2067c6f51086\") " pod="openshift-console/console-6dbcc9bbc4-gjtwp"
Oct 30 21:43:08.164579 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:08.164445    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c4xx8\" (UniqueName: \"kubernetes.io/projected/4460b81d-e403-41f8-a83a-2067c6f51086-kube-api-access-c4xx8\") pod \"console-6dbcc9bbc4-gjtwp\" (UID: \"4460b81d-e403-41f8-a83a-2067c6f51086\") " pod="openshift-console/console-6dbcc9bbc4-gjtwp"
Oct 30 21:43:08.164579 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:08.164489    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4460b81d-e403-41f8-a83a-2067c6f51086-service-ca\") pod \"console-6dbcc9bbc4-gjtwp\" (UID: \"4460b81d-e403-41f8-a83a-2067c6f51086\") " pod="openshift-console/console-6dbcc9bbc4-gjtwp"
Oct 30 21:43:08.165445 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:08.165096    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4460b81d-e403-41f8-a83a-2067c6f51086-oauth-serving-cert\") pod \"console-6dbcc9bbc4-gjtwp\" (UID: \"4460b81d-e403-41f8-a83a-2067c6f51086\") " pod="openshift-console/console-6dbcc9bbc4-gjtwp"
Oct 30 21:43:08.165445 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:08.165142    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4460b81d-e403-41f8-a83a-2067c6f51086-console-config\") pod \"console-6dbcc9bbc4-gjtwp\" (UID: \"4460b81d-e403-41f8-a83a-2067c6f51086\") " pod="openshift-console/console-6dbcc9bbc4-gjtwp"
Oct 30 21:43:08.165445 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:08.165145    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4460b81d-e403-41f8-a83a-2067c6f51086-service-ca\") pod \"console-6dbcc9bbc4-gjtwp\" (UID: \"4460b81d-e403-41f8-a83a-2067c6f51086\") " pod="openshift-console/console-6dbcc9bbc4-gjtwp"
Oct 30 21:43:08.165445 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:08.165373    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4460b81d-e403-41f8-a83a-2067c6f51086-tmp\") pod \"console-6dbcc9bbc4-gjtwp\" (UID: \"4460b81d-e403-41f8-a83a-2067c6f51086\") " pod="openshift-console/console-6dbcc9bbc4-gjtwp"
Oct 30 21:43:08.165981 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:08.165911    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4460b81d-e403-41f8-a83a-2067c6f51086-trusted-ca-bundle\") pod \"console-6dbcc9bbc4-gjtwp\" (UID: \"4460b81d-e403-41f8-a83a-2067c6f51086\") " pod="openshift-console/console-6dbcc9bbc4-gjtwp"
Oct 30 21:43:08.167527 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:08.167504    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4460b81d-e403-41f8-a83a-2067c6f51086-console-serving-cert\") pod \"console-6dbcc9bbc4-gjtwp\" (UID: \"4460b81d-e403-41f8-a83a-2067c6f51086\") " pod="openshift-console/console-6dbcc9bbc4-gjtwp"
Oct 30 21:43:08.167604 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:08.167530    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4460b81d-e403-41f8-a83a-2067c6f51086-console-oauth-config\") pod \"console-6dbcc9bbc4-gjtwp\" (UID: \"4460b81d-e403-41f8-a83a-2067c6f51086\") " pod="openshift-console/console-6dbcc9bbc4-gjtwp"
Oct 30 21:43:08.223999 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:08.223961    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c4xx8\" (UniqueName: \"kubernetes.io/projected/4460b81d-e403-41f8-a83a-2067c6f51086-kube-api-access-c4xx8\") pod \"console-6dbcc9bbc4-gjtwp\" (UID: \"4460b81d-e403-41f8-a83a-2067c6f51086\") " pod="openshift-console/console-6dbcc9bbc4-gjtwp"
Oct 30 21:43:08.338404 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:08.338320    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6dbcc9bbc4-gjtwp"
Oct 30 21:43:08.378491 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:08.378467    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Oct 30 21:43:08.380550 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:43:08.380518    4473 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc570a4d1_5a01_4beb_a659_1ac75ea3e768.slice/crio-24053fcde39d2b39d9d2a6415f5c7b703931fd1bb9aa6bf33d205dc12670e105 WatchSource:0}: Error finding container 24053fcde39d2b39d9d2a6415f5c7b703931fd1bb9aa6bf33d205dc12670e105: Status 404 returned error can't find the container with id 24053fcde39d2b39d9d2a6415f5c7b703931fd1bb9aa6bf33d205dc12670e105
Oct 30 21:43:08.427086 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:08.427062    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-68886bc5bd-pbpz9"]
Oct 30 21:43:08.428849 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:43:08.428820    4473 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd17e003_7b46_4f0d_b4e6_fad470cb81e8.slice/crio-c2bcc238bd5cc6569fa841f81a4441060f120cb8f8a847746dfa3b3dec11ff2b WatchSource:0}: Error finding container c2bcc238bd5cc6569fa841f81a4441060f120cb8f8a847746dfa3b3dec11ff2b: Status 404 returned error can't find the container with id c2bcc238bd5cc6569fa841f81a4441060f120cb8f8a847746dfa3b3dec11ff2b
Oct 30 21:43:08.510651 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:08.510615    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-b2pn2" event={"ID":"a1c5b288-d0e3-4e50-a9eb-f2d2660b3718","Type":"ContainerStarted","Data":"6c629c2675d8710a19e49c83d2fe83818318bb669b252e1629e14f49f25e1588"}
Oct 30 21:43:08.511137 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:08.510690    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-b2pn2" event={"ID":"a1c5b288-d0e3-4e50-a9eb-f2d2660b3718","Type":"ContainerStarted","Data":"7221f70999c716abc3833ba39adde38ddef3bd33e1ee3f10fed2f69ca65d59d1"}
Oct 30 21:43:08.511137 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:08.510714    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-b2pn2" event={"ID":"a1c5b288-d0e3-4e50-a9eb-f2d2660b3718","Type":"ContainerStarted","Data":"ddb384d91a56dd201fac2779c7ec3e1d52ddd20f47de01115e41757a27a1c92f"}
Oct 30 21:43:08.512766 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:08.512735    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-kzr98" event={"ID":"a20fdb9f-75a2-4373-9337-72fa18ffef81","Type":"ContainerStarted","Data":"cc687a8286dee5f0de4348a5494f6fdbe626f4e1f38a26fa2aa1f701302743b1"}
Oct 30 21:43:08.512901 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:08.512785    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-kzr98" event={"ID":"a20fdb9f-75a2-4373-9337-72fa18ffef81","Type":"ContainerStarted","Data":"70bd51d5c098f95f6b02321c2d291f78b1f137a76c104508163d460ce71e6b66"}
Oct 30 21:43:08.513986 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:08.513952    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c570a4d1-5a01-4beb-a659-1ac75ea3e768","Type":"ContainerStarted","Data":"24053fcde39d2b39d9d2a6415f5c7b703931fd1bb9aa6bf33d205dc12670e105"}
Oct 30 21:43:08.515023 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:08.514999    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-68886bc5bd-pbpz9" event={"ID":"dd17e003-7b46-4f0d-b4e6-fad470cb81e8","Type":"ContainerStarted","Data":"c2bcc238bd5cc6569fa841f81a4441060f120cb8f8a847746dfa3b3dec11ff2b"}
Oct 30 21:43:08.629670 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:08.629618    4473 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-b2pn2" podStartSLOduration=2.435850156 podStartE2EDuration="4.629600944s" totalImagesPullingTime="2.193750788s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:43:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:43:08.629454681 +0000 UTC m=+34.402823167" watchObservedRunningTime="2026-10-30 21:43:08.629600944 +0000 UTC m=+34.402969458"
Oct 30 21:43:08.725867 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:08.725837    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6dbcc9bbc4-gjtwp"]
Oct 30 21:43:08.728929 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:43:08.728876    4473 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4460b81d_e403_41f8_a83a_2067c6f51086.slice/crio-f0423e44f2c6def476f1d6c3ac225bcaca8d5b797431fe245d533f5f032e7562 WatchSource:0}: Error finding container f0423e44f2c6def476f1d6c3ac225bcaca8d5b797431fe245d533f5f032e7562: Status 404 returned error can't find the container with id f0423e44f2c6def476f1d6c3ac225bcaca8d5b797431fe245d533f5f032e7562
Oct 30 21:43:08.750426 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:08.750383    4473 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-kzr98" podStartSLOduration=2.701417408 podStartE2EDuration="4.750368801s" totalImagesPullingTime="1.943884834s" totalInitContainerRuntime="105.066559ms" isStatefulPod=true podCreationTimestamp="2026-10-30 21:43:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:43:08.749823719 +0000 UTC m=+34.523192205" watchObservedRunningTime="2026-10-30 21:43:08.750368801 +0000 UTC m=+34.523737291"
Oct 30 21:43:09.522065 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:09.522030    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6dbcc9bbc4-gjtwp" event={"ID":"4460b81d-e403-41f8-a83a-2067c6f51086","Type":"ContainerStarted","Data":"819b2547830ccc96a2e760b4f4bc72f9e320475a3da138ff6f02d16014da2205"}
Oct 30 21:43:09.522531 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:09.522085    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6dbcc9bbc4-gjtwp" event={"ID":"4460b81d-e403-41f8-a83a-2067c6f51086","Type":"ContainerStarted","Data":"f0423e44f2c6def476f1d6c3ac225bcaca8d5b797431fe245d533f5f032e7562"}
Oct 30 21:43:09.645547 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:09.643580    4473 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/console-6dbcc9bbc4-gjtwp" podStartSLOduration=2.643560429 podStartE2EDuration="2.643560429s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:43:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:43:09.641138366 +0000 UTC m=+35.414506858" watchObservedRunningTime="2026-10-30 21:43:09.643560429 +0000 UTC m=+35.416928917"
Oct 30 21:43:09.697485 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:09.697449    4473 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-77f4f76594-jjbhq"]
Oct 30 21:43:09.700102 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:09.700075    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-77f4f76594-jjbhq"
Oct 30 21:43:09.757906 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:09.757863    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-77f4f76594-jjbhq"]
Oct 30 21:43:09.776656 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:09.776638    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-ts4nf\""
Oct 30 21:43:09.776731 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:09.776696    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Oct 30 21:43:09.780977 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:09.780952    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ceb2c618-a371-489e-b148-1ae43d54a3d3-monitoring-plugin-cert\") pod \"monitoring-plugin-77f4f76594-jjbhq\" (UID: \"ceb2c618-a371-489e-b148-1ae43d54a3d3\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-jjbhq"
Oct 30 21:43:09.781093 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:09.781019    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t59b4\" (UniqueName: \"kubernetes.io/projected/ceb2c618-a371-489e-b148-1ae43d54a3d3-kube-api-access-t59b4\") pod \"monitoring-plugin-77f4f76594-jjbhq\" (UID: \"ceb2c618-a371-489e-b148-1ae43d54a3d3\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-jjbhq"
Oct 30 21:43:09.881675 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:09.881649    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ceb2c618-a371-489e-b148-1ae43d54a3d3-monitoring-plugin-cert\") pod \"monitoring-plugin-77f4f76594-jjbhq\" (UID: \"ceb2c618-a371-489e-b148-1ae43d54a3d3\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-jjbhq"
Oct 30 21:43:09.881851 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:09.881713    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t59b4\" (UniqueName: \"kubernetes.io/projected/ceb2c618-a371-489e-b148-1ae43d54a3d3-kube-api-access-t59b4\") pod \"monitoring-plugin-77f4f76594-jjbhq\" (UID: \"ceb2c618-a371-489e-b148-1ae43d54a3d3\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-jjbhq"
Oct 30 21:43:09.884304 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:09.884280    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ceb2c618-a371-489e-b148-1ae43d54a3d3-monitoring-plugin-cert\") pod \"monitoring-plugin-77f4f76594-jjbhq\" (UID: \"ceb2c618-a371-489e-b148-1ae43d54a3d3\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-jjbhq"
Oct 30 21:43:09.943737 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:09.943705    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t59b4\" (UniqueName: \"kubernetes.io/projected/ceb2c618-a371-489e-b148-1ae43d54a3d3-kube-api-access-t59b4\") pod \"monitoring-plugin-77f4f76594-jjbhq\" (UID: \"ceb2c618-a371-489e-b148-1ae43d54a3d3\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-jjbhq"
Oct 30 21:43:09.962577 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:09.962534    4473 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-krcxf_06552eb8-7379-40df-b31c-a133e646afb9/konnectivity-agent/0.log" limit=1024
Oct 30 21:43:10.013272 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:10.013243    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-77f4f76594-jjbhq"
Oct 30 21:43:10.381088 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:10.380850    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-77f4f76594-jjbhq"]
Oct 30 21:43:10.384059 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:43:10.384023    4473 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podceb2c618_a371_489e_b148_1ae43d54a3d3.slice/crio-6c48f4b87b9d18d09af9080b02518c2a4479e28ff540d06225b781f5fe28fc1e WatchSource:0}: Error finding container 6c48f4b87b9d18d09af9080b02518c2a4479e28ff540d06225b781f5fe28fc1e: Status 404 returned error can't find the container with id 6c48f4b87b9d18d09af9080b02518c2a4479e28ff540d06225b781f5fe28fc1e
Oct 30 21:43:10.502544 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:10.502508    4473 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-krcxf_06552eb8-7379-40df-b31c-a133e646afb9/konnectivity-agent/0.log" limit=1024
Oct 30 21:43:10.525852 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:10.525815    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-77f4f76594-jjbhq" event={"ID":"ceb2c618-a371-489e-b148-1ae43d54a3d3","Type":"ContainerStarted","Data":"6c48f4b87b9d18d09af9080b02518c2a4479e28ff540d06225b781f5fe28fc1e"}
Oct 30 21:43:10.527343 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:10.527317    4473 generic.go:356] "Generic (PLEG): container finished" podID="c570a4d1-5a01-4beb-a659-1ac75ea3e768" containerID="e8563f4e66dc3881af914b74b03c357d9849ce9b67ed7ea889e472ca524230e7" exitCode=0
Oct 30 21:43:10.527437 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:10.527406    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c570a4d1-5a01-4beb-a659-1ac75ea3e768","Type":"ContainerDied","Data":"e8563f4e66dc3881af914b74b03c357d9849ce9b67ed7ea889e472ca524230e7"}
Oct 30 21:43:10.765259 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:10.765180    4473 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-krcxf_06552eb8-7379-40df-b31c-a133e646afb9/konnectivity-agent/0.log" limit=1024
Oct 30 21:43:12.536273 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:12.535944    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-77f4f76594-jjbhq" event={"ID":"ceb2c618-a371-489e-b148-1ae43d54a3d3","Type":"ContainerStarted","Data":"ec147e661e44c17c43a1db96cd222832218ad46b18f51e48ec91476ebf730b4a"}
Oct 30 21:43:12.536273 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:12.536045    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-77f4f76594-jjbhq"
Oct 30 21:43:12.538171 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:12.538090    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c570a4d1-5a01-4beb-a659-1ac75ea3e768","Type":"ContainerStarted","Data":"afc8e513226b883d9f14b81c0a39e22958c330aa231326dd8ca39c9b1e1ee351"}
Oct 30 21:43:12.541242 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:12.541219    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-68886bc5bd-pbpz9" event={"ID":"dd17e003-7b46-4f0d-b4e6-fad470cb81e8","Type":"ContainerStarted","Data":"44f22d7465f2d25571937fda77b277a61858d5673a51bc5d04a52086d6bac3d4"}
Oct 30 21:43:12.541379 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:12.541287    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-68886bc5bd-pbpz9" event={"ID":"dd17e003-7b46-4f0d-b4e6-fad470cb81e8","Type":"ContainerStarted","Data":"6d77e6ef35b1814ae0d839894ae006501c29dd0e9ddcdab4232c57a6a86e7ed6"}
Oct 30 21:43:12.541379 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:12.541328    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-68886bc5bd-pbpz9" event={"ID":"dd17e003-7b46-4f0d-b4e6-fad470cb81e8","Type":"ContainerStarted","Data":"f6d8f30e956e393a9a4936a70c81bdec6115b5db86240ae601b3b75cc9437a72"}
Oct 30 21:43:12.542346 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:12.542324    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-77f4f76594-jjbhq"
Oct 30 21:43:12.655640 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:12.655584    4473 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-77f4f76594-jjbhq" podStartSLOduration=1.673494072 podStartE2EDuration="3.655566298s" totalImagesPullingTime="1.982072226s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:43:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:43:12.654639718 +0000 UTC m=+38.428008203" watchObservedRunningTime="2026-10-30 21:43:12.655566298 +0000 UTC m=+38.428934785"
Oct 30 21:43:13.552646 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:13.552616    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-68886bc5bd-pbpz9" event={"ID":"dd17e003-7b46-4f0d-b4e6-fad470cb81e8","Type":"ContainerStarted","Data":"22a1cc1c04a8b7c3458b10a6706d4a1625c8e55d72c750ea11908183927cd723"}
Oct 30 21:43:13.553055 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:13.552689    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-68886bc5bd-pbpz9" event={"ID":"dd17e003-7b46-4f0d-b4e6-fad470cb81e8","Type":"ContainerStarted","Data":"4f38a406b443386849dae0255cc0ce56384aa557bba0650dcd2d8cfe4974e891"}
Oct 30 21:43:13.553055 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:13.552723    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-68886bc5bd-pbpz9" event={"ID":"dd17e003-7b46-4f0d-b4e6-fad470cb81e8","Type":"ContainerStarted","Data":"dc9743377430c98bdcca3423f08c949bbd10647087ef6d9d7df79dea73a4c80b"}
Oct 30 21:43:13.553055 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:13.552763    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-68886bc5bd-pbpz9"
Oct 30 21:43:13.556126 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:13.556102    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c570a4d1-5a01-4beb-a659-1ac75ea3e768","Type":"ContainerStarted","Data":"66286a6856294a63ddfaa22d609c84b6e5906d98ae9d0ba09b874c27ff13297c"}
Oct 30 21:43:13.556215 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:13.556179    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c570a4d1-5a01-4beb-a659-1ac75ea3e768","Type":"ContainerStarted","Data":"f7275c39e728501f8f0bcae2c877d72e456bbf0b60fbf9fb94865572cfc832c1"}
Oct 30 21:43:13.556253 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:13.556214    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c570a4d1-5a01-4beb-a659-1ac75ea3e768","Type":"ContainerStarted","Data":"afa044491852d471fe3b0988530c11346b561a06fad61748858586958c55bbc9"}
Oct 30 21:43:13.556253 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:13.556237    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c570a4d1-5a01-4beb-a659-1ac75ea3e768","Type":"ContainerStarted","Data":"6b0b1baeba17b4e095f141ee41ad011f23b1b46f56082ebdc1bc1fadf1809b21"}
Oct 30 21:43:13.687275 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:13.687227    4473 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-68886bc5bd-pbpz9" podStartSLOduration=2.059566149 podStartE2EDuration="6.68721077s" totalImagesPullingTime="4.627644621s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:43:07 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:43:13.685280058 +0000 UTC m=+39.458648545" watchObservedRunningTime="2026-10-30 21:43:13.68721077 +0000 UTC m=+39.460579254"
Oct 30 21:43:14.563095 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:14.563056    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c570a4d1-5a01-4beb-a659-1ac75ea3e768","Type":"ContainerStarted","Data":"ae900c17c87ccc62c744b8d1eee0f2733f1768070492e8b7285c88217918f065"}
Oct 30 21:43:14.698150 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:14.698083    4473 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.553956191 podStartE2EDuration="7.698068849s" totalImagesPullingTime="4.125162103s" totalInitContainerRuntime="18.950555ms" isStatefulPod=false podCreationTimestamp="2026-10-30 21:43:07 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:43:14.696846107 +0000 UTC m=+40.470214594" watchObservedRunningTime="2026-10-30 21:43:14.698068849 +0000 UTC m=+40.471437335"
Oct 30 21:43:18.031834 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:18.031788    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6dbcc9bbc4-gjtwp"]
Oct 30 21:43:18.082296 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:18.082263    4473 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-7d5854ff44-24kdj"]
Oct 30 21:43:18.084268 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:18.084252    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7d5854ff44-24kdj"
Oct 30 21:43:18.144334 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:18.144308    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7d5854ff44-24kdj"]
Oct 30 21:43:18.152616 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:18.152576    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8d4d3d50-05e7-464e-9594-423b596e4604-service-ca\") pod \"console-7d5854ff44-24kdj\" (UID: \"8d4d3d50-05e7-464e-9594-423b596e4604\") " pod="openshift-console/console-7d5854ff44-24kdj"
Oct 30 21:43:18.152752 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:18.152627    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6pgm5\" (UniqueName: \"kubernetes.io/projected/8d4d3d50-05e7-464e-9594-423b596e4604-kube-api-access-6pgm5\") pod \"console-7d5854ff44-24kdj\" (UID: \"8d4d3d50-05e7-464e-9594-423b596e4604\") " pod="openshift-console/console-7d5854ff44-24kdj"
Oct 30 21:43:18.152752 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:18.152700    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8d4d3d50-05e7-464e-9594-423b596e4604-console-config\") pod \"console-7d5854ff44-24kdj\" (UID: \"8d4d3d50-05e7-464e-9594-423b596e4604\") " pod="openshift-console/console-7d5854ff44-24kdj"
Oct 30 21:43:18.152752 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:18.152740    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8d4d3d50-05e7-464e-9594-423b596e4604-console-oauth-config\") pod \"console-7d5854ff44-24kdj\" (UID: \"8d4d3d50-05e7-464e-9594-423b596e4604\") " pod="openshift-console/console-7d5854ff44-24kdj"
Oct 30 21:43:18.152896 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:18.152764    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8d4d3d50-05e7-464e-9594-423b596e4604-oauth-serving-cert\") pod \"console-7d5854ff44-24kdj\" (UID: \"8d4d3d50-05e7-464e-9594-423b596e4604\") " pod="openshift-console/console-7d5854ff44-24kdj"
Oct 30 21:43:18.152896 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:18.152781    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8d4d3d50-05e7-464e-9594-423b596e4604-console-serving-cert\") pod \"console-7d5854ff44-24kdj\" (UID: \"8d4d3d50-05e7-464e-9594-423b596e4604\") " pod="openshift-console/console-7d5854ff44-24kdj"
Oct 30 21:43:18.153025 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:18.152944    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8d4d3d50-05e7-464e-9594-423b596e4604-tmp\") pod \"console-7d5854ff44-24kdj\" (UID: \"8d4d3d50-05e7-464e-9594-423b596e4604\") " pod="openshift-console/console-7d5854ff44-24kdj"
Oct 30 21:43:18.153025 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:18.152985    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8d4d3d50-05e7-464e-9594-423b596e4604-trusted-ca-bundle\") pod \"console-7d5854ff44-24kdj\" (UID: \"8d4d3d50-05e7-464e-9594-423b596e4604\") " pod="openshift-console/console-7d5854ff44-24kdj"
Oct 30 21:43:18.253632 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:18.253599    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8d4d3d50-05e7-464e-9594-423b596e4604-tmp\") pod \"console-7d5854ff44-24kdj\" (UID: \"8d4d3d50-05e7-464e-9594-423b596e4604\") " pod="openshift-console/console-7d5854ff44-24kdj"
Oct 30 21:43:18.253821 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:18.253639    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8d4d3d50-05e7-464e-9594-423b596e4604-trusted-ca-bundle\") pod \"console-7d5854ff44-24kdj\" (UID: \"8d4d3d50-05e7-464e-9594-423b596e4604\") " pod="openshift-console/console-7d5854ff44-24kdj"
Oct 30 21:43:18.253821 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:18.253678    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8d4d3d50-05e7-464e-9594-423b596e4604-service-ca\") pod \"console-7d5854ff44-24kdj\" (UID: \"8d4d3d50-05e7-464e-9594-423b596e4604\") " pod="openshift-console/console-7d5854ff44-24kdj"
Oct 30 21:43:18.253821 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:18.253698    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6pgm5\" (UniqueName: \"kubernetes.io/projected/8d4d3d50-05e7-464e-9594-423b596e4604-kube-api-access-6pgm5\") pod \"console-7d5854ff44-24kdj\" (UID: \"8d4d3d50-05e7-464e-9594-423b596e4604\") " pod="openshift-console/console-7d5854ff44-24kdj"
Oct 30 21:43:18.253821 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:18.253727    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8d4d3d50-05e7-464e-9594-423b596e4604-console-config\") pod \"console-7d5854ff44-24kdj\" (UID: \"8d4d3d50-05e7-464e-9594-423b596e4604\") " pod="openshift-console/console-7d5854ff44-24kdj"
Oct 30 21:43:18.253821 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:18.253759    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8d4d3d50-05e7-464e-9594-423b596e4604-console-oauth-config\") pod \"console-7d5854ff44-24kdj\" (UID: \"8d4d3d50-05e7-464e-9594-423b596e4604\") " pod="openshift-console/console-7d5854ff44-24kdj"
Oct 30 21:43:18.253821 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:18.253792    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8d4d3d50-05e7-464e-9594-423b596e4604-oauth-serving-cert\") pod \"console-7d5854ff44-24kdj\" (UID: \"8d4d3d50-05e7-464e-9594-423b596e4604\") " pod="openshift-console/console-7d5854ff44-24kdj"
Oct 30 21:43:18.253821 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:18.253815    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8d4d3d50-05e7-464e-9594-423b596e4604-console-serving-cert\") pod \"console-7d5854ff44-24kdj\" (UID: \"8d4d3d50-05e7-464e-9594-423b596e4604\") " pod="openshift-console/console-7d5854ff44-24kdj"
Oct 30 21:43:18.254214 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:18.254071    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8d4d3d50-05e7-464e-9594-423b596e4604-tmp\") pod \"console-7d5854ff44-24kdj\" (UID: \"8d4d3d50-05e7-464e-9594-423b596e4604\") " pod="openshift-console/console-7d5854ff44-24kdj"
Oct 30 21:43:18.254610 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:18.254583    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8d4d3d50-05e7-464e-9594-423b596e4604-service-ca\") pod \"console-7d5854ff44-24kdj\" (UID: \"8d4d3d50-05e7-464e-9594-423b596e4604\") " pod="openshift-console/console-7d5854ff44-24kdj"
Oct 30 21:43:18.254808 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:18.254787    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8d4d3d50-05e7-464e-9594-423b596e4604-oauth-serving-cert\") pod \"console-7d5854ff44-24kdj\" (UID: \"8d4d3d50-05e7-464e-9594-423b596e4604\") " pod="openshift-console/console-7d5854ff44-24kdj"
Oct 30 21:43:18.254976 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:18.254956    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8d4d3d50-05e7-464e-9594-423b596e4604-trusted-ca-bundle\") pod \"console-7d5854ff44-24kdj\" (UID: \"8d4d3d50-05e7-464e-9594-423b596e4604\") " pod="openshift-console/console-7d5854ff44-24kdj"
Oct 30 21:43:18.255159 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:18.255137    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8d4d3d50-05e7-464e-9594-423b596e4604-console-config\") pod \"console-7d5854ff44-24kdj\" (UID: \"8d4d3d50-05e7-464e-9594-423b596e4604\") " pod="openshift-console/console-7d5854ff44-24kdj"
Oct 30 21:43:18.256485 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:18.256457    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8d4d3d50-05e7-464e-9594-423b596e4604-console-oauth-config\") pod \"console-7d5854ff44-24kdj\" (UID: \"8d4d3d50-05e7-464e-9594-423b596e4604\") " pod="openshift-console/console-7d5854ff44-24kdj"
Oct 30 21:43:18.256667 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:18.256648    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8d4d3d50-05e7-464e-9594-423b596e4604-console-serving-cert\") pod \"console-7d5854ff44-24kdj\" (UID: \"8d4d3d50-05e7-464e-9594-423b596e4604\") " pod="openshift-console/console-7d5854ff44-24kdj"
Oct 30 21:43:18.311797 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:18.311727    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6pgm5\" (UniqueName: \"kubernetes.io/projected/8d4d3d50-05e7-464e-9594-423b596e4604-kube-api-access-6pgm5\") pod \"console-7d5854ff44-24kdj\" (UID: \"8d4d3d50-05e7-464e-9594-423b596e4604\") " pod="openshift-console/console-7d5854ff44-24kdj"
Oct 30 21:43:18.338571 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:18.338541    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6dbcc9bbc4-gjtwp"
Oct 30 21:43:18.395044 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:18.395010    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7d5854ff44-24kdj"
Oct 30 21:43:18.769620 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:18.769574    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7d5854ff44-24kdj"]
Oct 30 21:43:18.772219 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:43:18.772196    4473 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d4d3d50_05e7_464e_9594_423b596e4604.slice/crio-f612718d6c32853672594ac8a5497a2b55be65328cb034f907686f2ff70ea6e9 WatchSource:0}: Error finding container f612718d6c32853672594ac8a5497a2b55be65328cb034f907686f2ff70ea6e9: Status 404 returned error can't find the container with id f612718d6c32853672594ac8a5497a2b55be65328cb034f907686f2ff70ea6e9
Oct 30 21:43:19.569666 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:19.569640    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-68886bc5bd-pbpz9"
Oct 30 21:43:19.582735 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:19.582695    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7d5854ff44-24kdj" event={"ID":"8d4d3d50-05e7-464e-9594-423b596e4604","Type":"ContainerStarted","Data":"03596d78ea0f978a4116903447c23cf82b6b2304894ac09f51908a94b31b8a28"}
Oct 30 21:43:19.582879 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:19.582751    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7d5854ff44-24kdj" event={"ID":"8d4d3d50-05e7-464e-9594-423b596e4604","Type":"ContainerStarted","Data":"f612718d6c32853672594ac8a5497a2b55be65328cb034f907686f2ff70ea6e9"}
Oct 30 21:43:19.965037 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:19.964984    4473 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/console-7d5854ff44-24kdj" podStartSLOduration=1.9649670019999999 podStartE2EDuration="1.964967002s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:43:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:43:19.964457275 +0000 UTC m=+45.737825762" watchObservedRunningTime="2026-10-30 21:43:19.964967002 +0000 UTC m=+45.738335483"
Oct 30 21:43:20.022380 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:20.022309    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7d5854ff44-24kdj"]
Oct 30 21:43:20.066195 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:20.066163    4473 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-548c99d9-nf7hj"]
Oct 30 21:43:20.068555 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:20.068534    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-548c99d9-nf7hj"
Oct 30 21:43:20.128349 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:20.128314    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-548c99d9-nf7hj"]
Oct 30 21:43:20.171585 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:20.171555    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e53bbdd4-5b5b-4a9d-bd11-8796b18af1b3-console-serving-cert\") pod \"console-548c99d9-nf7hj\" (UID: \"e53bbdd4-5b5b-4a9d-bd11-8796b18af1b3\") " pod="openshift-console/console-548c99d9-nf7hj"
Oct 30 21:43:20.171774 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:20.171593    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e53bbdd4-5b5b-4a9d-bd11-8796b18af1b3-console-config\") pod \"console-548c99d9-nf7hj\" (UID: \"e53bbdd4-5b5b-4a9d-bd11-8796b18af1b3\") " pod="openshift-console/console-548c99d9-nf7hj"
Oct 30 21:43:20.171774 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:20.171698    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9tt4n\" (UniqueName: \"kubernetes.io/projected/e53bbdd4-5b5b-4a9d-bd11-8796b18af1b3-kube-api-access-9tt4n\") pod \"console-548c99d9-nf7hj\" (UID: \"e53bbdd4-5b5b-4a9d-bd11-8796b18af1b3\") " pod="openshift-console/console-548c99d9-nf7hj"
Oct 30 21:43:20.171873 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:20.171794    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e53bbdd4-5b5b-4a9d-bd11-8796b18af1b3-oauth-serving-cert\") pod \"console-548c99d9-nf7hj\" (UID: \"e53bbdd4-5b5b-4a9d-bd11-8796b18af1b3\") " pod="openshift-console/console-548c99d9-nf7hj"
Oct 30 21:43:20.171873 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:20.171827    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e53bbdd4-5b5b-4a9d-bd11-8796b18af1b3-service-ca\") pod \"console-548c99d9-nf7hj\" (UID: \"e53bbdd4-5b5b-4a9d-bd11-8796b18af1b3\") " pod="openshift-console/console-548c99d9-nf7hj"
Oct 30 21:43:20.171975 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:20.171898    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e53bbdd4-5b5b-4a9d-bd11-8796b18af1b3-trusted-ca-bundle\") pod \"console-548c99d9-nf7hj\" (UID: \"e53bbdd4-5b5b-4a9d-bd11-8796b18af1b3\") " pod="openshift-console/console-548c99d9-nf7hj"
Oct 30 21:43:20.171975 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:20.171968    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e53bbdd4-5b5b-4a9d-bd11-8796b18af1b3-tmp\") pod \"console-548c99d9-nf7hj\" (UID: \"e53bbdd4-5b5b-4a9d-bd11-8796b18af1b3\") " pod="openshift-console/console-548c99d9-nf7hj"
Oct 30 21:43:20.172040 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:20.172002    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e53bbdd4-5b5b-4a9d-bd11-8796b18af1b3-console-oauth-config\") pod \"console-548c99d9-nf7hj\" (UID: \"e53bbdd4-5b5b-4a9d-bd11-8796b18af1b3\") " pod="openshift-console/console-548c99d9-nf7hj"
Oct 30 21:43:20.272774 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:20.272696    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e53bbdd4-5b5b-4a9d-bd11-8796b18af1b3-service-ca\") pod \"console-548c99d9-nf7hj\" (UID: \"e53bbdd4-5b5b-4a9d-bd11-8796b18af1b3\") " pod="openshift-console/console-548c99d9-nf7hj"
Oct 30 21:43:20.272774 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:20.272750    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e53bbdd4-5b5b-4a9d-bd11-8796b18af1b3-trusted-ca-bundle\") pod \"console-548c99d9-nf7hj\" (UID: \"e53bbdd4-5b5b-4a9d-bd11-8796b18af1b3\") " pod="openshift-console/console-548c99d9-nf7hj"
Oct 30 21:43:20.273039 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:20.272883    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e53bbdd4-5b5b-4a9d-bd11-8796b18af1b3-tmp\") pod \"console-548c99d9-nf7hj\" (UID: \"e53bbdd4-5b5b-4a9d-bd11-8796b18af1b3\") " pod="openshift-console/console-548c99d9-nf7hj"
Oct 30 21:43:20.273039 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:20.272981    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e53bbdd4-5b5b-4a9d-bd11-8796b18af1b3-console-oauth-config\") pod \"console-548c99d9-nf7hj\" (UID: \"e53bbdd4-5b5b-4a9d-bd11-8796b18af1b3\") " pod="openshift-console/console-548c99d9-nf7hj"
Oct 30 21:43:20.273147 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:20.273043    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e53bbdd4-5b5b-4a9d-bd11-8796b18af1b3-console-serving-cert\") pod \"console-548c99d9-nf7hj\" (UID: \"e53bbdd4-5b5b-4a9d-bd11-8796b18af1b3\") " pod="openshift-console/console-548c99d9-nf7hj"
Oct 30 21:43:20.273187 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:20.273162    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e53bbdd4-5b5b-4a9d-bd11-8796b18af1b3-console-config\") pod \"console-548c99d9-nf7hj\" (UID: \"e53bbdd4-5b5b-4a9d-bd11-8796b18af1b3\") " pod="openshift-console/console-548c99d9-nf7hj"
Oct 30 21:43:20.273231 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:20.273196    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9tt4n\" (UniqueName: \"kubernetes.io/projected/e53bbdd4-5b5b-4a9d-bd11-8796b18af1b3-kube-api-access-9tt4n\") pod \"console-548c99d9-nf7hj\" (UID: \"e53bbdd4-5b5b-4a9d-bd11-8796b18af1b3\") " pod="openshift-console/console-548c99d9-nf7hj"
Oct 30 21:43:20.273271 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:20.273231    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e53bbdd4-5b5b-4a9d-bd11-8796b18af1b3-oauth-serving-cert\") pod \"console-548c99d9-nf7hj\" (UID: \"e53bbdd4-5b5b-4a9d-bd11-8796b18af1b3\") " pod="openshift-console/console-548c99d9-nf7hj"
Oct 30 21:43:20.273623 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:20.273572    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e53bbdd4-5b5b-4a9d-bd11-8796b18af1b3-service-ca\") pod \"console-548c99d9-nf7hj\" (UID: \"e53bbdd4-5b5b-4a9d-bd11-8796b18af1b3\") " pod="openshift-console/console-548c99d9-nf7hj"
Oct 30 21:43:20.273750 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:20.273667    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e53bbdd4-5b5b-4a9d-bd11-8796b18af1b3-tmp\") pod \"console-548c99d9-nf7hj\" (UID: \"e53bbdd4-5b5b-4a9d-bd11-8796b18af1b3\") " pod="openshift-console/console-548c99d9-nf7hj"
Oct 30 21:43:20.273954 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:20.273908    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e53bbdd4-5b5b-4a9d-bd11-8796b18af1b3-trusted-ca-bundle\") pod \"console-548c99d9-nf7hj\" (UID: \"e53bbdd4-5b5b-4a9d-bd11-8796b18af1b3\") " pod="openshift-console/console-548c99d9-nf7hj"
Oct 30 21:43:20.274073 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:20.274049    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e53bbdd4-5b5b-4a9d-bd11-8796b18af1b3-console-config\") pod \"console-548c99d9-nf7hj\" (UID: \"e53bbdd4-5b5b-4a9d-bd11-8796b18af1b3\") " pod="openshift-console/console-548c99d9-nf7hj"
Oct 30 21:43:20.274132 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:20.274056    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e53bbdd4-5b5b-4a9d-bd11-8796b18af1b3-oauth-serving-cert\") pod \"console-548c99d9-nf7hj\" (UID: \"e53bbdd4-5b5b-4a9d-bd11-8796b18af1b3\") " pod="openshift-console/console-548c99d9-nf7hj"
Oct 30 21:43:20.275743 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:20.275725    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e53bbdd4-5b5b-4a9d-bd11-8796b18af1b3-console-serving-cert\") pod \"console-548c99d9-nf7hj\" (UID: \"e53bbdd4-5b5b-4a9d-bd11-8796b18af1b3\") " pod="openshift-console/console-548c99d9-nf7hj"
Oct 30 21:43:20.275796 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:20.275735    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e53bbdd4-5b5b-4a9d-bd11-8796b18af1b3-console-oauth-config\") pod \"console-548c99d9-nf7hj\" (UID: \"e53bbdd4-5b5b-4a9d-bd11-8796b18af1b3\") " pod="openshift-console/console-548c99d9-nf7hj"
Oct 30 21:43:20.323152 ip-10-0-140-18 kubenswrapper[4473]: E1030 21:43:20.323117    4473 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fncvg" containerName="extractor"
Oct 30 21:43:20.338310 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:20.338286    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9tt4n\" (UniqueName: \"kubernetes.io/projected/e53bbdd4-5b5b-4a9d-bd11-8796b18af1b3-kube-api-access-9tt4n\") pod \"console-548c99d9-nf7hj\" (UID: \"e53bbdd4-5b5b-4a9d-bd11-8796b18af1b3\") " pod="openshift-console/console-548c99d9-nf7hj"
Oct 30 21:43:20.378832 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:20.378793    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-548c99d9-nf7hj"
Oct 30 21:43:20.752310 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:20.752280    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-548c99d9-nf7hj"]
Oct 30 21:43:20.755235 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:43:20.755211    4473 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode53bbdd4_5b5b_4a9d_bd11_8796b18af1b3.slice/crio-3a4e0f570161c1182a2f83ce69dea704b3e2ff2ed0da6e0a2da020c8c4f26bf5 WatchSource:0}: Error finding container 3a4e0f570161c1182a2f83ce69dea704b3e2ff2ed0da6e0a2da020c8c4f26bf5: Status 404 returned error can't find the container with id 3a4e0f570161c1182a2f83ce69dea704b3e2ff2ed0da6e0a2da020c8c4f26bf5
Oct 30 21:43:21.592316 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:21.592285    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-548c99d9-nf7hj" event={"ID":"e53bbdd4-5b5b-4a9d-bd11-8796b18af1b3","Type":"ContainerStarted","Data":"7402d5acb284aba4de6964ccc4ca9c52b23dbca9b3f7f10f965f2a05637fa79e"}
Oct 30 21:43:21.592477 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:21.592332    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-548c99d9-nf7hj" event={"ID":"e53bbdd4-5b5b-4a9d-bd11-8796b18af1b3","Type":"ContainerStarted","Data":"3a4e0f570161c1182a2f83ce69dea704b3e2ff2ed0da6e0a2da020c8c4f26bf5"}
Oct 30 21:43:21.882684 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:21.882633    4473 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/console-548c99d9-nf7hj" podStartSLOduration=1.882618678 podStartE2EDuration="1.882618678s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:43:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:43:21.881366268 +0000 UTC m=+47.654734759" watchObservedRunningTime="2026-10-30 21:43:21.882618678 +0000 UTC m=+47.655987164"
Oct 30 21:43:28.395311 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:28.395266    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7d5854ff44-24kdj"
Oct 30 21:43:30.379402 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:30.379363    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-548c99d9-nf7hj"
Oct 30 21:43:30.379826 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:30.379637    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-548c99d9-nf7hj"
Oct 30 21:43:30.384483 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:30.384460    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-548c99d9-nf7hj"
Oct 30 21:43:30.627026 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:30.627001    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-548c99d9-nf7hj"
Oct 30 21:43:30.792939 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:30.792841    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-57d5867d6f-94q6b"]
Oct 30 21:43:43.053310 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:43.053242    4473 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-console/console-6dbcc9bbc4-gjtwp" podUID="4460b81d-e403-41f8-a83a-2067c6f51086" containerName="console" containerID="cri-o://819b2547830ccc96a2e760b4f4bc72f9e320475a3da138ff6f02d16014da2205" gracePeriod=15
Oct 30 21:43:43.331713 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:43.331632    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6dbcc9bbc4-gjtwp"
Oct 30 21:43:43.372341 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:43.372301    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4460b81d-e403-41f8-a83a-2067c6f51086-console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4460b81d-e403-41f8-a83a-2067c6f51086-console-serving-cert\") pod \"4460b81d-e403-41f8-a83a-2067c6f51086\" (UID: \"4460b81d-e403-41f8-a83a-2067c6f51086\") "
Oct 30 21:43:43.372494 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:43.372353    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4460b81d-e403-41f8-a83a-2067c6f51086-kube-api-access-c4xx8\" (UniqueName: \"kubernetes.io/projected/4460b81d-e403-41f8-a83a-2067c6f51086-kube-api-access-c4xx8\") pod \"4460b81d-e403-41f8-a83a-2067c6f51086\" (UID: \"4460b81d-e403-41f8-a83a-2067c6f51086\") "
Oct 30 21:43:43.372494 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:43.372378    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4460b81d-e403-41f8-a83a-2067c6f51086-console-config\" (UniqueName: \"kubernetes.io/configmap/4460b81d-e403-41f8-a83a-2067c6f51086-console-config\") pod \"4460b81d-e403-41f8-a83a-2067c6f51086\" (UID: \"4460b81d-e403-41f8-a83a-2067c6f51086\") "
Oct 30 21:43:43.372494 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:43.372398    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4460b81d-e403-41f8-a83a-2067c6f51086-console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4460b81d-e403-41f8-a83a-2067c6f51086-console-oauth-config\") pod \"4460b81d-e403-41f8-a83a-2067c6f51086\" (UID: \"4460b81d-e403-41f8-a83a-2067c6f51086\") "
Oct 30 21:43:43.372494 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:43.372417    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4460b81d-e403-41f8-a83a-2067c6f51086-tmp\" (UniqueName: \"kubernetes.io/empty-dir/4460b81d-e403-41f8-a83a-2067c6f51086-tmp\") pod \"4460b81d-e403-41f8-a83a-2067c6f51086\" (UID: \"4460b81d-e403-41f8-a83a-2067c6f51086\") "
Oct 30 21:43:43.372707 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:43.372506    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4460b81d-e403-41f8-a83a-2067c6f51086-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4460b81d-e403-41f8-a83a-2067c6f51086-trusted-ca-bundle\") pod \"4460b81d-e403-41f8-a83a-2067c6f51086\" (UID: \"4460b81d-e403-41f8-a83a-2067c6f51086\") "
Oct 30 21:43:43.372707 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:43.372548    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4460b81d-e403-41f8-a83a-2067c6f51086-oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4460b81d-e403-41f8-a83a-2067c6f51086-oauth-serving-cert\") pod \"4460b81d-e403-41f8-a83a-2067c6f51086\" (UID: \"4460b81d-e403-41f8-a83a-2067c6f51086\") "
Oct 30 21:43:43.372707 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:43.372583    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4460b81d-e403-41f8-a83a-2067c6f51086-service-ca\" (UniqueName: \"kubernetes.io/configmap/4460b81d-e403-41f8-a83a-2067c6f51086-service-ca\") pod \"4460b81d-e403-41f8-a83a-2067c6f51086\" (UID: \"4460b81d-e403-41f8-a83a-2067c6f51086\") "
Oct 30 21:43:43.372853 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:43.372785    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4460b81d-e403-41f8-a83a-2067c6f51086-console-config" pod "4460b81d-e403-41f8-a83a-2067c6f51086" (UID: "4460b81d-e403-41f8-a83a-2067c6f51086"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:43:43.373127 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:43.373096    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4460b81d-e403-41f8-a83a-2067c6f51086-tmp" pod "4460b81d-e403-41f8-a83a-2067c6f51086" (UID: "4460b81d-e403-41f8-a83a-2067c6f51086"). InnerVolumeSpecName "tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:43:43.373230 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:43.373152    4473 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4460b81d-e403-41f8-a83a-2067c6f51086-console-config\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:43:43.373230 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:43.373168    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4460b81d-e403-41f8-a83a-2067c6f51086-trusted-ca-bundle" pod "4460b81d-e403-41f8-a83a-2067c6f51086" (UID: "4460b81d-e403-41f8-a83a-2067c6f51086"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:43:43.373230 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:43.373196    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4460b81d-e403-41f8-a83a-2067c6f51086-service-ca" pod "4460b81d-e403-41f8-a83a-2067c6f51086" (UID: "4460b81d-e403-41f8-a83a-2067c6f51086"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:43:43.373561 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:43.373535    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4460b81d-e403-41f8-a83a-2067c6f51086-oauth-serving-cert" pod "4460b81d-e403-41f8-a83a-2067c6f51086" (UID: "4460b81d-e403-41f8-a83a-2067c6f51086"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:43:43.374929 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:43.374894    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4460b81d-e403-41f8-a83a-2067c6f51086-console-oauth-config" pod "4460b81d-e403-41f8-a83a-2067c6f51086" (UID: "4460b81d-e403-41f8-a83a-2067c6f51086"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:43:43.375232 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:43.375211    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4460b81d-e403-41f8-a83a-2067c6f51086-console-serving-cert" pod "4460b81d-e403-41f8-a83a-2067c6f51086" (UID: "4460b81d-e403-41f8-a83a-2067c6f51086"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:43:43.375297 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:43.375270    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4460b81d-e403-41f8-a83a-2067c6f51086-kube-api-access-c4xx8" pod "4460b81d-e403-41f8-a83a-2067c6f51086" (UID: "4460b81d-e403-41f8-a83a-2067c6f51086"). InnerVolumeSpecName "kube-api-access-c4xx8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:43:43.473955 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:43.473903    4473 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4460b81d-e403-41f8-a83a-2067c6f51086-console-serving-cert\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:43:43.473955 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:43.473954    4473 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c4xx8\" (UniqueName: \"kubernetes.io/projected/4460b81d-e403-41f8-a83a-2067c6f51086-kube-api-access-c4xx8\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:43:43.474158 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:43.473976    4473 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4460b81d-e403-41f8-a83a-2067c6f51086-console-oauth-config\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:43:43.474158 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:43.473989    4473 reconciler_common.go:299] "Volume detached for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4460b81d-e403-41f8-a83a-2067c6f51086-tmp\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:43:43.474158 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:43.474002    4473 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4460b81d-e403-41f8-a83a-2067c6f51086-trusted-ca-bundle\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:43:43.474158 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:43.474013    4473 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4460b81d-e403-41f8-a83a-2067c6f51086-oauth-serving-cert\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:43:43.474158 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:43.474024    4473 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4460b81d-e403-41f8-a83a-2067c6f51086-service-ca\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:43:43.667395 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:43.667363    4473 generic.go:356] "Generic (PLEG): container finished" podID="4460b81d-e403-41f8-a83a-2067c6f51086" containerID="819b2547830ccc96a2e760b4f4bc72f9e320475a3da138ff6f02d16014da2205" exitCode=0
Oct 30 21:43:43.667571 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:43.667404    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6dbcc9bbc4-gjtwp"
Oct 30 21:43:43.667571 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:43.667417    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6dbcc9bbc4-gjtwp" event={"ID":"4460b81d-e403-41f8-a83a-2067c6f51086","Type":"ContainerDied","Data":"819b2547830ccc96a2e760b4f4bc72f9e320475a3da138ff6f02d16014da2205"}
Oct 30 21:43:43.667571 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:43.667463    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6dbcc9bbc4-gjtwp" event={"ID":"4460b81d-e403-41f8-a83a-2067c6f51086","Type":"ContainerDied","Data":"f0423e44f2c6def476f1d6c3ac225bcaca8d5b797431fe245d533f5f032e7562"}
Oct 30 21:43:43.667571 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:43.667483    4473 scope.go:122] "RemoveContainer" containerID="819b2547830ccc96a2e760b4f4bc72f9e320475a3da138ff6f02d16014da2205"
Oct 30 21:43:43.676431 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:43.676412    4473 scope.go:122] "RemoveContainer" containerID="819b2547830ccc96a2e760b4f4bc72f9e320475a3da138ff6f02d16014da2205"
Oct 30 21:43:43.676763 ip-10-0-140-18 kubenswrapper[4473]: E1030 21:43:43.676743    4473 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"819b2547830ccc96a2e760b4f4bc72f9e320475a3da138ff6f02d16014da2205\": container with ID starting with 819b2547830ccc96a2e760b4f4bc72f9e320475a3da138ff6f02d16014da2205 not found: ID does not exist" containerID="819b2547830ccc96a2e760b4f4bc72f9e320475a3da138ff6f02d16014da2205"
Oct 30 21:43:43.676833 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:43.676772    4473 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"819b2547830ccc96a2e760b4f4bc72f9e320475a3da138ff6f02d16014da2205"} err="failed to get container status \"819b2547830ccc96a2e760b4f4bc72f9e320475a3da138ff6f02d16014da2205\": rpc error: code = NotFound desc = could not find container \"819b2547830ccc96a2e760b4f4bc72f9e320475a3da138ff6f02d16014da2205\": container with ID starting with 819b2547830ccc96a2e760b4f4bc72f9e320475a3da138ff6f02d16014da2205 not found: ID does not exist"
Oct 30 21:43:43.836530 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:43.836495    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6dbcc9bbc4-gjtwp"]
Oct 30 21:43:43.840707 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:43.840681    4473 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6dbcc9bbc4-gjtwp"]
Oct 30 21:43:44.354565 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:44.354532    4473 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4460b81d-e403-41f8-a83a-2067c6f51086" path="/var/lib/kubelet/pods/4460b81d-e403-41f8-a83a-2067c6f51086/volumes"
Oct 30 21:43:46.611685 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:46.611649    4473 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-console/console-7d5854ff44-24kdj" podUID="8d4d3d50-05e7-464e-9594-423b596e4604" containerName="console" containerID="cri-o://03596d78ea0f978a4116903447c23cf82b6b2304894ac09f51908a94b31b8a28" gracePeriod=15
Oct 30 21:43:46.864665 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:46.864646    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7d5854ff44-24kdj"
Oct 30 21:43:46.904452 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:46.904420    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8d4d3d50-05e7-464e-9594-423b596e4604-service-ca\" (UniqueName: \"kubernetes.io/configmap/8d4d3d50-05e7-464e-9594-423b596e4604-service-ca\") pod \"8d4d3d50-05e7-464e-9594-423b596e4604\" (UID: \"8d4d3d50-05e7-464e-9594-423b596e4604\") "
Oct 30 21:43:46.904452 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:46.904458    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8d4d3d50-05e7-464e-9594-423b596e4604-console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8d4d3d50-05e7-464e-9594-423b596e4604-console-serving-cert\") pod \"8d4d3d50-05e7-464e-9594-423b596e4604\" (UID: \"8d4d3d50-05e7-464e-9594-423b596e4604\") "
Oct 30 21:43:46.904678 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:46.904492    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8d4d3d50-05e7-464e-9594-423b596e4604-kube-api-access-6pgm5\" (UniqueName: \"kubernetes.io/projected/8d4d3d50-05e7-464e-9594-423b596e4604-kube-api-access-6pgm5\") pod \"8d4d3d50-05e7-464e-9594-423b596e4604\" (UID: \"8d4d3d50-05e7-464e-9594-423b596e4604\") "
Oct 30 21:43:46.904678 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:46.904519    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8d4d3d50-05e7-464e-9594-423b596e4604-console-config\" (UniqueName: \"kubernetes.io/configmap/8d4d3d50-05e7-464e-9594-423b596e4604-console-config\") pod \"8d4d3d50-05e7-464e-9594-423b596e4604\" (UID: \"8d4d3d50-05e7-464e-9594-423b596e4604\") "
Oct 30 21:43:46.904678 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:46.904538    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8d4d3d50-05e7-464e-9594-423b596e4604-console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8d4d3d50-05e7-464e-9594-423b596e4604-console-oauth-config\") pod \"8d4d3d50-05e7-464e-9594-423b596e4604\" (UID: \"8d4d3d50-05e7-464e-9594-423b596e4604\") "
Oct 30 21:43:46.904678 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:46.904560    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8d4d3d50-05e7-464e-9594-423b596e4604-oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8d4d3d50-05e7-464e-9594-423b596e4604-oauth-serving-cert\") pod \"8d4d3d50-05e7-464e-9594-423b596e4604\" (UID: \"8d4d3d50-05e7-464e-9594-423b596e4604\") "
Oct 30 21:43:46.904678 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:46.904586    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8d4d3d50-05e7-464e-9594-423b596e4604-tmp\" (UniqueName: \"kubernetes.io/empty-dir/8d4d3d50-05e7-464e-9594-423b596e4604-tmp\") pod \"8d4d3d50-05e7-464e-9594-423b596e4604\" (UID: \"8d4d3d50-05e7-464e-9594-423b596e4604\") "
Oct 30 21:43:46.904678 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:46.904659    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8d4d3d50-05e7-464e-9594-423b596e4604-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8d4d3d50-05e7-464e-9594-423b596e4604-trusted-ca-bundle\") pod \"8d4d3d50-05e7-464e-9594-423b596e4604\" (UID: \"8d4d3d50-05e7-464e-9594-423b596e4604\") "
Oct 30 21:43:46.905001 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:46.904866    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8d4d3d50-05e7-464e-9594-423b596e4604-service-ca" pod "8d4d3d50-05e7-464e-9594-423b596e4604" (UID: "8d4d3d50-05e7-464e-9594-423b596e4604"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:43:46.905057 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:46.905009    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8d4d3d50-05e7-464e-9594-423b596e4604-console-config" pod "8d4d3d50-05e7-464e-9594-423b596e4604" (UID: "8d4d3d50-05e7-464e-9594-423b596e4604"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:43:46.905057 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:46.905004    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8d4d3d50-05e7-464e-9594-423b596e4604-tmp" pod "8d4d3d50-05e7-464e-9594-423b596e4604" (UID: "8d4d3d50-05e7-464e-9594-423b596e4604"). InnerVolumeSpecName "tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:43:46.905250 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:46.905216    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8d4d3d50-05e7-464e-9594-423b596e4604-trusted-ca-bundle" pod "8d4d3d50-05e7-464e-9594-423b596e4604" (UID: "8d4d3d50-05e7-464e-9594-423b596e4604"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:43:46.905250 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:46.905230    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8d4d3d50-05e7-464e-9594-423b596e4604-oauth-serving-cert" pod "8d4d3d50-05e7-464e-9594-423b596e4604" (UID: "8d4d3d50-05e7-464e-9594-423b596e4604"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:43:46.906826 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:46.906800    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8d4d3d50-05e7-464e-9594-423b596e4604-console-serving-cert" pod "8d4d3d50-05e7-464e-9594-423b596e4604" (UID: "8d4d3d50-05e7-464e-9594-423b596e4604"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:43:46.906947 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:46.906841    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8d4d3d50-05e7-464e-9594-423b596e4604-console-oauth-config" pod "8d4d3d50-05e7-464e-9594-423b596e4604" (UID: "8d4d3d50-05e7-464e-9594-423b596e4604"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:43:46.906947 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:46.906932    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8d4d3d50-05e7-464e-9594-423b596e4604-kube-api-access-6pgm5" pod "8d4d3d50-05e7-464e-9594-423b596e4604" (UID: "8d4d3d50-05e7-464e-9594-423b596e4604"). InnerVolumeSpecName "kube-api-access-6pgm5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:43:47.005270 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:47.005241    4473 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8d4d3d50-05e7-464e-9594-423b596e4604-console-config\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:43:47.005270 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:47.005268    4473 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8d4d3d50-05e7-464e-9594-423b596e4604-console-oauth-config\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:43:47.005270 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:47.005277    4473 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8d4d3d50-05e7-464e-9594-423b596e4604-oauth-serving-cert\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:43:47.005537 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:47.005287    4473 reconciler_common.go:299] "Volume detached for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8d4d3d50-05e7-464e-9594-423b596e4604-tmp\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:43:47.005537 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:47.005296    4473 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8d4d3d50-05e7-464e-9594-423b596e4604-trusted-ca-bundle\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:43:47.005537 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:47.005304    4473 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8d4d3d50-05e7-464e-9594-423b596e4604-service-ca\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:43:47.005537 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:47.005312    4473 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8d4d3d50-05e7-464e-9594-423b596e4604-console-serving-cert\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:43:47.005537 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:47.005322    4473 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6pgm5\" (UniqueName: \"kubernetes.io/projected/8d4d3d50-05e7-464e-9594-423b596e4604-kube-api-access-6pgm5\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:43:47.684288 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:47.684254    4473 generic.go:356] "Generic (PLEG): container finished" podID="8d4d3d50-05e7-464e-9594-423b596e4604" containerID="03596d78ea0f978a4116903447c23cf82b6b2304894ac09f51908a94b31b8a28" exitCode=0
Oct 30 21:43:47.684723 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:47.684296    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7d5854ff44-24kdj"
Oct 30 21:43:47.684723 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:47.684317    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7d5854ff44-24kdj" event={"ID":"8d4d3d50-05e7-464e-9594-423b596e4604","Type":"ContainerDied","Data":"03596d78ea0f978a4116903447c23cf82b6b2304894ac09f51908a94b31b8a28"}
Oct 30 21:43:47.684723 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:47.684356    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7d5854ff44-24kdj" event={"ID":"8d4d3d50-05e7-464e-9594-423b596e4604","Type":"ContainerDied","Data":"f612718d6c32853672594ac8a5497a2b55be65328cb034f907686f2ff70ea6e9"}
Oct 30 21:43:47.684723 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:47.684377    4473 scope.go:122] "RemoveContainer" containerID="03596d78ea0f978a4116903447c23cf82b6b2304894ac09f51908a94b31b8a28"
Oct 30 21:43:47.693435 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:47.693420    4473 scope.go:122] "RemoveContainer" containerID="03596d78ea0f978a4116903447c23cf82b6b2304894ac09f51908a94b31b8a28"
Oct 30 21:43:47.693689 ip-10-0-140-18 kubenswrapper[4473]: E1030 21:43:47.693668    4473 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"03596d78ea0f978a4116903447c23cf82b6b2304894ac09f51908a94b31b8a28\": container with ID starting with 03596d78ea0f978a4116903447c23cf82b6b2304894ac09f51908a94b31b8a28 not found: ID does not exist" containerID="03596d78ea0f978a4116903447c23cf82b6b2304894ac09f51908a94b31b8a28"
Oct 30 21:43:47.693752 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:47.693702    4473 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"03596d78ea0f978a4116903447c23cf82b6b2304894ac09f51908a94b31b8a28"} err="failed to get container status \"03596d78ea0f978a4116903447c23cf82b6b2304894ac09f51908a94b31b8a28\": rpc error: code = NotFound desc = could not find container \"03596d78ea0f978a4116903447c23cf82b6b2304894ac09f51908a94b31b8a28\": container with ID starting with 03596d78ea0f978a4116903447c23cf82b6b2304894ac09f51908a94b31b8a28 not found: ID does not exist"
Oct 30 21:43:47.851269 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:47.851234    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7d5854ff44-24kdj"]
Oct 30 21:43:47.901824 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:47.901790    4473 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7d5854ff44-24kdj"]
Oct 30 21:43:48.357549 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:48.355994    4473 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8d4d3d50-05e7-464e-9594-423b596e4604" path="/var/lib/kubelet/pods/8d4d3d50-05e7-464e-9594-423b596e4604/volumes"
Oct 30 21:43:50.323026 ip-10-0-140-18 kubenswrapper[4473]: E1030 21:43:50.322990    4473 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fncvg" containerName="extractor"
Oct 30 21:43:55.813185 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:55.813120    4473 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-console/console-57d5867d6f-94q6b" podUID="60cba30b-6c5f-4409-a108-dc8f71232fd9" containerName="console" containerID="cri-o://80db523a46a63b09be44c755ade961a23f8379f91908ccd28065c4dfad827768" gracePeriod=15
Oct 30 21:43:56.058164 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:56.058140    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-57d5867d6f-94q6b"
Oct 30 21:43:56.181219 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:56.180949    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/60cba30b-6c5f-4409-a108-dc8f71232fd9-service-ca\" (UniqueName: \"kubernetes.io/configmap/60cba30b-6c5f-4409-a108-dc8f71232fd9-service-ca\") pod \"60cba30b-6c5f-4409-a108-dc8f71232fd9\" (UID: \"60cba30b-6c5f-4409-a108-dc8f71232fd9\") "
Oct 30 21:43:56.181586 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:56.181375    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/60cba30b-6c5f-4409-a108-dc8f71232fd9-service-ca" pod "60cba30b-6c5f-4409-a108-dc8f71232fd9" (UID: "60cba30b-6c5f-4409-a108-dc8f71232fd9"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:43:56.181586 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:56.181572    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/60cba30b-6c5f-4409-a108-dc8f71232fd9-kube-api-access-jq66c\" (UniqueName: \"kubernetes.io/projected/60cba30b-6c5f-4409-a108-dc8f71232fd9-kube-api-access-jq66c\") pod \"60cba30b-6c5f-4409-a108-dc8f71232fd9\" (UID: \"60cba30b-6c5f-4409-a108-dc8f71232fd9\") "
Oct 30 21:43:56.181881 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:56.181633    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/60cba30b-6c5f-4409-a108-dc8f71232fd9-console-config\" (UniqueName: \"kubernetes.io/configmap/60cba30b-6c5f-4409-a108-dc8f71232fd9-console-config\") pod \"60cba30b-6c5f-4409-a108-dc8f71232fd9\" (UID: \"60cba30b-6c5f-4409-a108-dc8f71232fd9\") "
Oct 30 21:43:56.181881 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:56.181658    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/60cba30b-6c5f-4409-a108-dc8f71232fd9-tmp\" (UniqueName: \"kubernetes.io/empty-dir/60cba30b-6c5f-4409-a108-dc8f71232fd9-tmp\") pod \"60cba30b-6c5f-4409-a108-dc8f71232fd9\" (UID: \"60cba30b-6c5f-4409-a108-dc8f71232fd9\") "
Oct 30 21:43:56.181881 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:56.181692    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/60cba30b-6c5f-4409-a108-dc8f71232fd9-console-oauth-config\" (UniqueName: \"kubernetes.io/secret/60cba30b-6c5f-4409-a108-dc8f71232fd9-console-oauth-config\") pod \"60cba30b-6c5f-4409-a108-dc8f71232fd9\" (UID: \"60cba30b-6c5f-4409-a108-dc8f71232fd9\") "
Oct 30 21:43:56.181881 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:56.181757    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/60cba30b-6c5f-4409-a108-dc8f71232fd9-console-serving-cert\" (UniqueName: \"kubernetes.io/secret/60cba30b-6c5f-4409-a108-dc8f71232fd9-console-serving-cert\") pod \"60cba30b-6c5f-4409-a108-dc8f71232fd9\" (UID: \"60cba30b-6c5f-4409-a108-dc8f71232fd9\") "
Oct 30 21:43:56.181881 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:56.181803    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/60cba30b-6c5f-4409-a108-dc8f71232fd9-oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/60cba30b-6c5f-4409-a108-dc8f71232fd9-oauth-serving-cert\") pod \"60cba30b-6c5f-4409-a108-dc8f71232fd9\" (UID: \"60cba30b-6c5f-4409-a108-dc8f71232fd9\") "
Oct 30 21:43:56.182189 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:56.182044    4473 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/60cba30b-6c5f-4409-a108-dc8f71232fd9-service-ca\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:43:56.182308 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:56.182263    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60cba30b-6c5f-4409-a108-dc8f71232fd9-tmp" pod "60cba30b-6c5f-4409-a108-dc8f71232fd9" (UID: "60cba30b-6c5f-4409-a108-dc8f71232fd9"). InnerVolumeSpecName "tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:43:56.182585 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:56.182331    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/60cba30b-6c5f-4409-a108-dc8f71232fd9-oauth-serving-cert" pod "60cba30b-6c5f-4409-a108-dc8f71232fd9" (UID: "60cba30b-6c5f-4409-a108-dc8f71232fd9"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:43:56.182585 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:56.182384    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/60cba30b-6c5f-4409-a108-dc8f71232fd9-console-config" pod "60cba30b-6c5f-4409-a108-dc8f71232fd9" (UID: "60cba30b-6c5f-4409-a108-dc8f71232fd9"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:43:56.184020 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:56.183991    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/60cba30b-6c5f-4409-a108-dc8f71232fd9-kube-api-access-jq66c" pod "60cba30b-6c5f-4409-a108-dc8f71232fd9" (UID: "60cba30b-6c5f-4409-a108-dc8f71232fd9"). InnerVolumeSpecName "kube-api-access-jq66c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:43:56.184227 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:56.184204    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/60cba30b-6c5f-4409-a108-dc8f71232fd9-console-serving-cert" pod "60cba30b-6c5f-4409-a108-dc8f71232fd9" (UID: "60cba30b-6c5f-4409-a108-dc8f71232fd9"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:43:56.184428 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:56.184403    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/60cba30b-6c5f-4409-a108-dc8f71232fd9-console-oauth-config" pod "60cba30b-6c5f-4409-a108-dc8f71232fd9" (UID: "60cba30b-6c5f-4409-a108-dc8f71232fd9"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:43:56.282981 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:56.282946    4473 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/60cba30b-6c5f-4409-a108-dc8f71232fd9-oauth-serving-cert\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:43:56.282981 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:56.282975    4473 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jq66c\" (UniqueName: \"kubernetes.io/projected/60cba30b-6c5f-4409-a108-dc8f71232fd9-kube-api-access-jq66c\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:43:56.282981 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:56.282986    4473 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/60cba30b-6c5f-4409-a108-dc8f71232fd9-console-config\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:43:56.283208 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:56.282996    4473 reconciler_common.go:299] "Volume detached for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/60cba30b-6c5f-4409-a108-dc8f71232fd9-tmp\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:43:56.283208 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:56.283005    4473 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/60cba30b-6c5f-4409-a108-dc8f71232fd9-console-oauth-config\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:43:56.283208 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:56.283013    4473 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/60cba30b-6c5f-4409-a108-dc8f71232fd9-console-serving-cert\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:43:56.556377 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:56.556300    4473 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-7c5745455d-z9bjx"]
Oct 30 21:43:56.556595 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:56.556583    4473 cpu_manager.go:414] "containerMap: removing container" podUID="4460b81d-e403-41f8-a83a-2067c6f51086" containerName="console"
Oct 30 21:43:56.556635 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:56.556597    4473 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4460b81d-e403-41f8-a83a-2067c6f51086" containerName="console"
Oct 30 21:43:56.556635 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:56.556607    4473 cpu_manager.go:414] "containerMap: removing container" podUID="8d4d3d50-05e7-464e-9594-423b596e4604" containerName="console"
Oct 30 21:43:56.556635 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:56.556614    4473 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8d4d3d50-05e7-464e-9594-423b596e4604" containerName="console"
Oct 30 21:43:56.556635 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:56.556625    4473 cpu_manager.go:414] "containerMap: removing container" podUID="60cba30b-6c5f-4409-a108-dc8f71232fd9" containerName="console"
Oct 30 21:43:56.556635 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:56.556630    4473 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="60cba30b-6c5f-4409-a108-dc8f71232fd9" containerName="console"
Oct 30 21:43:56.556776 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:56.556698    4473 memory_manager.go:365] "RemoveStaleState removing state" podUID="4460b81d-e403-41f8-a83a-2067c6f51086" containerName="console"
Oct 30 21:43:56.556776 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:56.556704    4473 memory_manager.go:365] "RemoveStaleState removing state" podUID="8d4d3d50-05e7-464e-9594-423b596e4604" containerName="console"
Oct 30 21:43:56.556776 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:56.556710    4473 memory_manager.go:365] "RemoveStaleState removing state" podUID="60cba30b-6c5f-4409-a108-dc8f71232fd9" containerName="console"
Oct 30 21:43:56.558612 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:56.558597    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c5745455d-z9bjx"
Oct 30 21:43:56.660268 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:56.660235    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7c5745455d-z9bjx"]
Oct 30 21:43:56.684711 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:56.684681    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1fa79e6f-8777-4e11-8392-6227556e2593-oauth-serving-cert\") pod \"console-7c5745455d-z9bjx\" (UID: \"1fa79e6f-8777-4e11-8392-6227556e2593\") " pod="openshift-console/console-7c5745455d-z9bjx"
Oct 30 21:43:56.684711 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:56.684719    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1fa79e6f-8777-4e11-8392-6227556e2593-console-serving-cert\") pod \"console-7c5745455d-z9bjx\" (UID: \"1fa79e6f-8777-4e11-8392-6227556e2593\") " pod="openshift-console/console-7c5745455d-z9bjx"
Oct 30 21:43:56.684913 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:56.684737    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1fa79e6f-8777-4e11-8392-6227556e2593-trusted-ca-bundle\") pod \"console-7c5745455d-z9bjx\" (UID: \"1fa79e6f-8777-4e11-8392-6227556e2593\") " pod="openshift-console/console-7c5745455d-z9bjx"
Oct 30 21:43:56.684913 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:56.684814    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1fa79e6f-8777-4e11-8392-6227556e2593-console-oauth-config\") pod \"console-7c5745455d-z9bjx\" (UID: \"1fa79e6f-8777-4e11-8392-6227556e2593\") " pod="openshift-console/console-7c5745455d-z9bjx"
Oct 30 21:43:56.684913 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:56.684866    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hzcxw\" (UniqueName: \"kubernetes.io/projected/1fa79e6f-8777-4e11-8392-6227556e2593-kube-api-access-hzcxw\") pod \"console-7c5745455d-z9bjx\" (UID: \"1fa79e6f-8777-4e11-8392-6227556e2593\") " pod="openshift-console/console-7c5745455d-z9bjx"
Oct 30 21:43:56.684913 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:56.684900    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1fa79e6f-8777-4e11-8392-6227556e2593-tmp\") pod \"console-7c5745455d-z9bjx\" (UID: \"1fa79e6f-8777-4e11-8392-6227556e2593\") " pod="openshift-console/console-7c5745455d-z9bjx"
Oct 30 21:43:56.685060 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:56.684961    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1fa79e6f-8777-4e11-8392-6227556e2593-console-config\") pod \"console-7c5745455d-z9bjx\" (UID: \"1fa79e6f-8777-4e11-8392-6227556e2593\") " pod="openshift-console/console-7c5745455d-z9bjx"
Oct 30 21:43:56.685060 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:56.684977    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1fa79e6f-8777-4e11-8392-6227556e2593-service-ca\") pod \"console-7c5745455d-z9bjx\" (UID: \"1fa79e6f-8777-4e11-8392-6227556e2593\") " pod="openshift-console/console-7c5745455d-z9bjx"
Oct 30 21:43:56.718959 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:56.718900    4473 generic.go:356] "Generic (PLEG): container finished" podID="60cba30b-6c5f-4409-a108-dc8f71232fd9" containerID="80db523a46a63b09be44c755ade961a23f8379f91908ccd28065c4dfad827768" exitCode=0
Oct 30 21:43:56.719113 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:56.718978    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-57d5867d6f-94q6b"
Oct 30 21:43:56.719113 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:56.719010    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-57d5867d6f-94q6b" event={"ID":"60cba30b-6c5f-4409-a108-dc8f71232fd9","Type":"ContainerDied","Data":"80db523a46a63b09be44c755ade961a23f8379f91908ccd28065c4dfad827768"}
Oct 30 21:43:56.719113 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:56.719061    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-57d5867d6f-94q6b" event={"ID":"60cba30b-6c5f-4409-a108-dc8f71232fd9","Type":"ContainerDied","Data":"d50fa3f5b0231adccdeeffba3b75731efb86bf04a8003c9f48a39b67763bf5ba"}
Oct 30 21:43:56.719113 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:56.719082    4473 scope.go:122] "RemoveContainer" containerID="80db523a46a63b09be44c755ade961a23f8379f91908ccd28065c4dfad827768"
Oct 30 21:43:56.726886 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:56.726864    4473 scope.go:122] "RemoveContainer" containerID="80db523a46a63b09be44c755ade961a23f8379f91908ccd28065c4dfad827768"
Oct 30 21:43:56.727198 ip-10-0-140-18 kubenswrapper[4473]: E1030 21:43:56.727179    4473 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"80db523a46a63b09be44c755ade961a23f8379f91908ccd28065c4dfad827768\": container with ID starting with 80db523a46a63b09be44c755ade961a23f8379f91908ccd28065c4dfad827768 not found: ID does not exist" containerID="80db523a46a63b09be44c755ade961a23f8379f91908ccd28065c4dfad827768"
Oct 30 21:43:56.727259 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:56.727206    4473 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"80db523a46a63b09be44c755ade961a23f8379f91908ccd28065c4dfad827768"} err="failed to get container status \"80db523a46a63b09be44c755ade961a23f8379f91908ccd28065c4dfad827768\": rpc error: code = NotFound desc = could not find container \"80db523a46a63b09be44c755ade961a23f8379f91908ccd28065c4dfad827768\": container with ID starting with 80db523a46a63b09be44c755ade961a23f8379f91908ccd28065c4dfad827768 not found: ID does not exist"
Oct 30 21:43:56.786402 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:56.786370    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1fa79e6f-8777-4e11-8392-6227556e2593-oauth-serving-cert\") pod \"console-7c5745455d-z9bjx\" (UID: \"1fa79e6f-8777-4e11-8392-6227556e2593\") " pod="openshift-console/console-7c5745455d-z9bjx"
Oct 30 21:43:56.786575 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:56.786409    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1fa79e6f-8777-4e11-8392-6227556e2593-console-serving-cert\") pod \"console-7c5745455d-z9bjx\" (UID: \"1fa79e6f-8777-4e11-8392-6227556e2593\") " pod="openshift-console/console-7c5745455d-z9bjx"
Oct 30 21:43:56.786575 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:56.786430    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1fa79e6f-8777-4e11-8392-6227556e2593-trusted-ca-bundle\") pod \"console-7c5745455d-z9bjx\" (UID: \"1fa79e6f-8777-4e11-8392-6227556e2593\") " pod="openshift-console/console-7c5745455d-z9bjx"
Oct 30 21:43:56.786575 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:56.786554    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1fa79e6f-8777-4e11-8392-6227556e2593-console-oauth-config\") pod \"console-7c5745455d-z9bjx\" (UID: \"1fa79e6f-8777-4e11-8392-6227556e2593\") " pod="openshift-console/console-7c5745455d-z9bjx"
Oct 30 21:43:56.786740 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:56.786611    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hzcxw\" (UniqueName: \"kubernetes.io/projected/1fa79e6f-8777-4e11-8392-6227556e2593-kube-api-access-hzcxw\") pod \"console-7c5745455d-z9bjx\" (UID: \"1fa79e6f-8777-4e11-8392-6227556e2593\") " pod="openshift-console/console-7c5745455d-z9bjx"
Oct 30 21:43:56.786740 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:56.786663    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1fa79e6f-8777-4e11-8392-6227556e2593-tmp\") pod \"console-7c5745455d-z9bjx\" (UID: \"1fa79e6f-8777-4e11-8392-6227556e2593\") " pod="openshift-console/console-7c5745455d-z9bjx"
Oct 30 21:43:56.786740 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:56.786701    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1fa79e6f-8777-4e11-8392-6227556e2593-console-config\") pod \"console-7c5745455d-z9bjx\" (UID: \"1fa79e6f-8777-4e11-8392-6227556e2593\") " pod="openshift-console/console-7c5745455d-z9bjx"
Oct 30 21:43:56.786740 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:56.786726    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1fa79e6f-8777-4e11-8392-6227556e2593-service-ca\") pod \"console-7c5745455d-z9bjx\" (UID: \"1fa79e6f-8777-4e11-8392-6227556e2593\") " pod="openshift-console/console-7c5745455d-z9bjx"
Oct 30 21:43:56.787143 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:56.787089    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1fa79e6f-8777-4e11-8392-6227556e2593-tmp\") pod \"console-7c5745455d-z9bjx\" (UID: \"1fa79e6f-8777-4e11-8392-6227556e2593\") " pod="openshift-console/console-7c5745455d-z9bjx"
Oct 30 21:43:56.787420 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:56.787254    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1fa79e6f-8777-4e11-8392-6227556e2593-trusted-ca-bundle\") pod \"console-7c5745455d-z9bjx\" (UID: \"1fa79e6f-8777-4e11-8392-6227556e2593\") " pod="openshift-console/console-7c5745455d-z9bjx"
Oct 30 21:43:56.787420 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:56.787332    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1fa79e6f-8777-4e11-8392-6227556e2593-service-ca\") pod \"console-7c5745455d-z9bjx\" (UID: \"1fa79e6f-8777-4e11-8392-6227556e2593\") " pod="openshift-console/console-7c5745455d-z9bjx"
Oct 30 21:43:56.787550 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:56.787532    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1fa79e6f-8777-4e11-8392-6227556e2593-console-config\") pod \"console-7c5745455d-z9bjx\" (UID: \"1fa79e6f-8777-4e11-8392-6227556e2593\") " pod="openshift-console/console-7c5745455d-z9bjx"
Oct 30 21:43:56.787735 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:56.787560    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1fa79e6f-8777-4e11-8392-6227556e2593-oauth-serving-cert\") pod \"console-7c5745455d-z9bjx\" (UID: \"1fa79e6f-8777-4e11-8392-6227556e2593\") " pod="openshift-console/console-7c5745455d-z9bjx"
Oct 30 21:43:56.789090 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:56.789073    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1fa79e6f-8777-4e11-8392-6227556e2593-console-serving-cert\") pod \"console-7c5745455d-z9bjx\" (UID: \"1fa79e6f-8777-4e11-8392-6227556e2593\") " pod="openshift-console/console-7c5745455d-z9bjx"
Oct 30 21:43:56.789169 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:56.789098    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1fa79e6f-8777-4e11-8392-6227556e2593-console-oauth-config\") pod \"console-7c5745455d-z9bjx\" (UID: \"1fa79e6f-8777-4e11-8392-6227556e2593\") " pod="openshift-console/console-7c5745455d-z9bjx"
Oct 30 21:43:56.844576 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:56.844512    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hzcxw\" (UniqueName: \"kubernetes.io/projected/1fa79e6f-8777-4e11-8392-6227556e2593-kube-api-access-hzcxw\") pod \"console-7c5745455d-z9bjx\" (UID: \"1fa79e6f-8777-4e11-8392-6227556e2593\") " pod="openshift-console/console-7c5745455d-z9bjx"
Oct 30 21:43:56.868406 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:56.868378    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c5745455d-z9bjx"
Oct 30 21:43:56.889465 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:56.889435    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-57d5867d6f-94q6b"]
Oct 30 21:43:56.892542 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:56.892517    4473 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-57d5867d6f-94q6b"]
Oct 30 21:43:57.246451 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:57.246428    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7c5745455d-z9bjx"]
Oct 30 21:43:57.249118 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:43:57.249092    4473 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1fa79e6f_8777_4e11_8392_6227556e2593.slice/crio-7e748d50ec3cdf1159734e356e7a97dc8e6b52351470ac7e6a165c71648f7b6b WatchSource:0}: Error finding container 7e748d50ec3cdf1159734e356e7a97dc8e6b52351470ac7e6a165c71648f7b6b: Status 404 returned error can't find the container with id 7e748d50ec3cdf1159734e356e7a97dc8e6b52351470ac7e6a165c71648f7b6b
Oct 30 21:43:57.729159 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:57.729122    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c5745455d-z9bjx" event={"ID":"1fa79e6f-8777-4e11-8392-6227556e2593","Type":"ContainerStarted","Data":"e84e67b2b04547dbf973eb4fa3a5c208cd8a68469bcc0ce80f761a158072c888"}
Oct 30 21:43:57.729365 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:57.729192    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c5745455d-z9bjx" event={"ID":"1fa79e6f-8777-4e11-8392-6227556e2593","Type":"ContainerStarted","Data":"7e748d50ec3cdf1159734e356e7a97dc8e6b52351470ac7e6a165c71648f7b6b"}
Oct 30 21:43:57.860764 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:57.860709    4473 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/console-7c5745455d-z9bjx" podStartSLOduration=1.860694993 podStartE2EDuration="1.860694993s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:43:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:43:57.860234121 +0000 UTC m=+83.633602607" watchObservedRunningTime="2026-10-30 21:43:57.860694993 +0000 UTC m=+83.634063480"
Oct 30 21:43:58.354586 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:43:58.354551    4473 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="60cba30b-6c5f-4409-a108-dc8f71232fd9" path="/var/lib/kubelet/pods/60cba30b-6c5f-4409-a108-dc8f71232fd9/volumes"
Oct 30 21:44:00.507164 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:00.507134    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Oct 30 21:44:00.507691 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:00.507447    4473 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="c570a4d1-5a01-4beb-a659-1ac75ea3e768" containerName="alertmanager" containerID="cri-o://afc8e513226b883d9f14b81c0a39e22958c330aa231326dd8ca39c9b1e1ee351" gracePeriod=120
Oct 30 21:44:00.507691 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:00.507498    4473 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="c570a4d1-5a01-4beb-a659-1ac75ea3e768" containerName="kube-rbac-proxy-metric" containerID="cri-o://66286a6856294a63ddfaa22d609c84b6e5906d98ae9d0ba09b874c27ff13297c" gracePeriod=120
Oct 30 21:44:00.507691 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:00.507573    4473 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="c570a4d1-5a01-4beb-a659-1ac75ea3e768" containerName="kube-rbac-proxy-web" containerID="cri-o://afa044491852d471fe3b0988530c11346b561a06fad61748858586958c55bbc9" gracePeriod=120
Oct 30 21:44:00.507691 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:00.507578    4473 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="c570a4d1-5a01-4beb-a659-1ac75ea3e768" containerName="config-reloader" containerID="cri-o://6b0b1baeba17b4e095f141ee41ad011f23b1b46f56082ebdc1bc1fadf1809b21" gracePeriod=120
Oct 30 21:44:00.507691 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:00.507554    4473 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="c570a4d1-5a01-4beb-a659-1ac75ea3e768" containerName="prom-label-proxy" containerID="cri-o://ae900c17c87ccc62c744b8d1eee0f2733f1768070492e8b7285c88217918f065" gracePeriod=120
Oct 30 21:44:00.507691 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:00.507591    4473 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="c570a4d1-5a01-4beb-a659-1ac75ea3e768" containerName="kube-rbac-proxy" containerID="cri-o://f7275c39e728501f8f0bcae2c877d72e456bbf0b60fbf9fb94865572cfc832c1" gracePeriod=120
Oct 30 21:44:00.743279 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:00.743245    4473 generic.go:356] "Generic (PLEG): container finished" podID="c570a4d1-5a01-4beb-a659-1ac75ea3e768" containerID="ae900c17c87ccc62c744b8d1eee0f2733f1768070492e8b7285c88217918f065" exitCode=0
Oct 30 21:44:00.743279 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:00.743271    4473 generic.go:356] "Generic (PLEG): container finished" podID="c570a4d1-5a01-4beb-a659-1ac75ea3e768" containerID="f7275c39e728501f8f0bcae2c877d72e456bbf0b60fbf9fb94865572cfc832c1" exitCode=0
Oct 30 21:44:00.743279 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:00.743278    4473 generic.go:356] "Generic (PLEG): container finished" podID="c570a4d1-5a01-4beb-a659-1ac75ea3e768" containerID="6b0b1baeba17b4e095f141ee41ad011f23b1b46f56082ebdc1bc1fadf1809b21" exitCode=0
Oct 30 21:44:00.743279 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:00.743284    4473 generic.go:356] "Generic (PLEG): container finished" podID="c570a4d1-5a01-4beb-a659-1ac75ea3e768" containerID="afc8e513226b883d9f14b81c0a39e22958c330aa231326dd8ca39c9b1e1ee351" exitCode=0
Oct 30 21:44:00.743525 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:00.743311    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c570a4d1-5a01-4beb-a659-1ac75ea3e768","Type":"ContainerDied","Data":"ae900c17c87ccc62c744b8d1eee0f2733f1768070492e8b7285c88217918f065"}
Oct 30 21:44:00.743525 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:00.743373    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c570a4d1-5a01-4beb-a659-1ac75ea3e768","Type":"ContainerDied","Data":"f7275c39e728501f8f0bcae2c877d72e456bbf0b60fbf9fb94865572cfc832c1"}
Oct 30 21:44:00.743525 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:00.743397    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c570a4d1-5a01-4beb-a659-1ac75ea3e768","Type":"ContainerDied","Data":"6b0b1baeba17b4e095f141ee41ad011f23b1b46f56082ebdc1bc1fadf1809b21"}
Oct 30 21:44:00.743525 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:00.743421    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c570a4d1-5a01-4beb-a659-1ac75ea3e768","Type":"ContainerDied","Data":"afc8e513226b883d9f14b81c0a39e22958c330aa231326dd8ca39c9b1e1ee351"}
Oct 30 21:44:01.750314 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:01.750283    4473 generic.go:356] "Generic (PLEG): container finished" podID="c570a4d1-5a01-4beb-a659-1ac75ea3e768" containerID="66286a6856294a63ddfaa22d609c84b6e5906d98ae9d0ba09b874c27ff13297c" exitCode=0
Oct 30 21:44:01.750314 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:01.750309    4473 generic.go:356] "Generic (PLEG): container finished" podID="c570a4d1-5a01-4beb-a659-1ac75ea3e768" containerID="afa044491852d471fe3b0988530c11346b561a06fad61748858586958c55bbc9" exitCode=0
Oct 30 21:44:01.750709 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:01.750347    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c570a4d1-5a01-4beb-a659-1ac75ea3e768","Type":"ContainerDied","Data":"66286a6856294a63ddfaa22d609c84b6e5906d98ae9d0ba09b874c27ff13297c"}
Oct 30 21:44:01.750709 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:01.750443    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c570a4d1-5a01-4beb-a659-1ac75ea3e768","Type":"ContainerDied","Data":"afa044491852d471fe3b0988530c11346b561a06fad61748858586958c55bbc9"}
Oct 30 21:44:01.750709 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:01.750471    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c570a4d1-5a01-4beb-a659-1ac75ea3e768","Type":"ContainerDied","Data":"24053fcde39d2b39d9d2a6415f5c7b703931fd1bb9aa6bf33d205dc12670e105"}
Oct 30 21:44:01.750709 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:01.750524    4473 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="24053fcde39d2b39d9d2a6415f5c7b703931fd1bb9aa6bf33d205dc12670e105"
Oct 30 21:44:01.753508 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:01.753490    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:44:01.831653 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:01.831578    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c570a4d1-5a01-4beb-a659-1ac75ea3e768-web-config\" (UniqueName: \"kubernetes.io/secret/c570a4d1-5a01-4beb-a659-1ac75ea3e768-web-config\") pod \"c570a4d1-5a01-4beb-a659-1ac75ea3e768\" (UID: \"c570a4d1-5a01-4beb-a659-1ac75ea3e768\") "
Oct 30 21:44:01.831653 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:01.831625    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c570a4d1-5a01-4beb-a659-1ac75ea3e768-cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/c570a4d1-5a01-4beb-a659-1ac75ea3e768-cluster-tls-config\") pod \"c570a4d1-5a01-4beb-a659-1ac75ea3e768\" (UID: \"c570a4d1-5a01-4beb-a659-1ac75ea3e768\") "
Oct 30 21:44:01.831861 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:01.831666    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c570a4d1-5a01-4beb-a659-1ac75ea3e768-kube-api-access-bz72j\" (UniqueName: \"kubernetes.io/projected/c570a4d1-5a01-4beb-a659-1ac75ea3e768-kube-api-access-bz72j\") pod \"c570a4d1-5a01-4beb-a659-1ac75ea3e768\" (UID: \"c570a4d1-5a01-4beb-a659-1ac75ea3e768\") "
Oct 30 21:44:01.831861 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:01.831700    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c570a4d1-5a01-4beb-a659-1ac75ea3e768-secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/c570a4d1-5a01-4beb-a659-1ac75ea3e768-secret-alertmanager-main-tls\") pod \"c570a4d1-5a01-4beb-a659-1ac75ea3e768\" (UID: \"c570a4d1-5a01-4beb-a659-1ac75ea3e768\") "
Oct 30 21:44:01.831861 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:01.831734    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c570a4d1-5a01-4beb-a659-1ac75ea3e768-alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c570a4d1-5a01-4beb-a659-1ac75ea3e768-alertmanager-trusted-ca-bundle\") pod \"c570a4d1-5a01-4beb-a659-1ac75ea3e768\" (UID: \"c570a4d1-5a01-4beb-a659-1ac75ea3e768\") "
Oct 30 21:44:01.831861 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:01.831772    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c570a4d1-5a01-4beb-a659-1ac75ea3e768-alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/c570a4d1-5a01-4beb-a659-1ac75ea3e768-alertmanager-main-db\") pod \"c570a4d1-5a01-4beb-a659-1ac75ea3e768\" (UID: \"c570a4d1-5a01-4beb-a659-1ac75ea3e768\") "
Oct 30 21:44:01.831861 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:01.831817    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c570a4d1-5a01-4beb-a659-1ac75ea3e768-tls-assets\" (UniqueName: \"kubernetes.io/projected/c570a4d1-5a01-4beb-a659-1ac75ea3e768-tls-assets\") pod \"c570a4d1-5a01-4beb-a659-1ac75ea3e768\" (UID: \"c570a4d1-5a01-4beb-a659-1ac75ea3e768\") "
Oct 30 21:44:01.832163 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:01.831850    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c570a4d1-5a01-4beb-a659-1ac75ea3e768-config-out\" (UniqueName: \"kubernetes.io/empty-dir/c570a4d1-5a01-4beb-a659-1ac75ea3e768-config-out\") pod \"c570a4d1-5a01-4beb-a659-1ac75ea3e768\" (UID: \"c570a4d1-5a01-4beb-a659-1ac75ea3e768\") "
Oct 30 21:44:01.832163 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:01.831945    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c570a4d1-5a01-4beb-a659-1ac75ea3e768-secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/c570a4d1-5a01-4beb-a659-1ac75ea3e768-secret-alertmanager-kube-rbac-proxy-metric\") pod \"c570a4d1-5a01-4beb-a659-1ac75ea3e768\" (UID: \"c570a4d1-5a01-4beb-a659-1ac75ea3e768\") "
Oct 30 21:44:01.832163 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:01.832022    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c570a4d1-5a01-4beb-a659-1ac75ea3e768-secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c570a4d1-5a01-4beb-a659-1ac75ea3e768-secret-alertmanager-kube-rbac-proxy-web\") pod \"c570a4d1-5a01-4beb-a659-1ac75ea3e768\" (UID: \"c570a4d1-5a01-4beb-a659-1ac75ea3e768\") "
Oct 30 21:44:01.832163 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:01.832052    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c570a4d1-5a01-4beb-a659-1ac75ea3e768-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c570a4d1-5a01-4beb-a659-1ac75ea3e768-metrics-client-ca\") pod \"c570a4d1-5a01-4beb-a659-1ac75ea3e768\" (UID: \"c570a4d1-5a01-4beb-a659-1ac75ea3e768\") "
Oct 30 21:44:01.832163 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:01.832083    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c570a4d1-5a01-4beb-a659-1ac75ea3e768-secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c570a4d1-5a01-4beb-a659-1ac75ea3e768-secret-alertmanager-kube-rbac-proxy\") pod \"c570a4d1-5a01-4beb-a659-1ac75ea3e768\" (UID: \"c570a4d1-5a01-4beb-a659-1ac75ea3e768\") "
Oct 30 21:44:01.832163 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:01.832112    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c570a4d1-5a01-4beb-a659-1ac75ea3e768-config-volume\" (UniqueName: \"kubernetes.io/secret/c570a4d1-5a01-4beb-a659-1ac75ea3e768-config-volume\") pod \"c570a4d1-5a01-4beb-a659-1ac75ea3e768\" (UID: \"c570a4d1-5a01-4beb-a659-1ac75ea3e768\") "
Oct 30 21:44:01.832651 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:01.832212    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c570a4d1-5a01-4beb-a659-1ac75ea3e768-alertmanager-main-db" pod "c570a4d1-5a01-4beb-a659-1ac75ea3e768" (UID: "c570a4d1-5a01-4beb-a659-1ac75ea3e768"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:44:01.832651 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:01.832526    4473 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/c570a4d1-5a01-4beb-a659-1ac75ea3e768-alertmanager-main-db\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:44:01.832807 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:01.832715    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c570a4d1-5a01-4beb-a659-1ac75ea3e768-alertmanager-trusted-ca-bundle" pod "c570a4d1-5a01-4beb-a659-1ac75ea3e768" (UID: "c570a4d1-5a01-4beb-a659-1ac75ea3e768"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:44:01.833502 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:01.833239    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c570a4d1-5a01-4beb-a659-1ac75ea3e768-metrics-client-ca" pod "c570a4d1-5a01-4beb-a659-1ac75ea3e768" (UID: "c570a4d1-5a01-4beb-a659-1ac75ea3e768"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:44:01.834968 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:01.834938    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c570a4d1-5a01-4beb-a659-1ac75ea3e768-kube-api-access-bz72j" pod "c570a4d1-5a01-4beb-a659-1ac75ea3e768" (UID: "c570a4d1-5a01-4beb-a659-1ac75ea3e768"). InnerVolumeSpecName "kube-api-access-bz72j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:44:01.835285 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:01.835250    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c570a4d1-5a01-4beb-a659-1ac75ea3e768-secret-alertmanager-main-tls" pod "c570a4d1-5a01-4beb-a659-1ac75ea3e768" (UID: "c570a4d1-5a01-4beb-a659-1ac75ea3e768"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:44:01.835387 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:01.835324    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c570a4d1-5a01-4beb-a659-1ac75ea3e768-secret-alertmanager-kube-rbac-proxy-metric" pod "c570a4d1-5a01-4beb-a659-1ac75ea3e768" (UID: "c570a4d1-5a01-4beb-a659-1ac75ea3e768"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:44:01.835599 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:01.835576    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c570a4d1-5a01-4beb-a659-1ac75ea3e768-secret-alertmanager-kube-rbac-proxy-web" pod "c570a4d1-5a01-4beb-a659-1ac75ea3e768" (UID: "c570a4d1-5a01-4beb-a659-1ac75ea3e768"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:44:01.835903 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:01.835869    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c570a4d1-5a01-4beb-a659-1ac75ea3e768-config-volume" pod "c570a4d1-5a01-4beb-a659-1ac75ea3e768" (UID: "c570a4d1-5a01-4beb-a659-1ac75ea3e768"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:44:01.836099 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:01.836066    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c570a4d1-5a01-4beb-a659-1ac75ea3e768-tls-assets" pod "c570a4d1-5a01-4beb-a659-1ac75ea3e768" (UID: "c570a4d1-5a01-4beb-a659-1ac75ea3e768"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:44:01.836721 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:01.836699    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c570a4d1-5a01-4beb-a659-1ac75ea3e768-secret-alertmanager-kube-rbac-proxy" pod "c570a4d1-5a01-4beb-a659-1ac75ea3e768" (UID: "c570a4d1-5a01-4beb-a659-1ac75ea3e768"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:44:01.836998 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:01.836980    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c570a4d1-5a01-4beb-a659-1ac75ea3e768-config-out" pod "c570a4d1-5a01-4beb-a659-1ac75ea3e768" (UID: "c570a4d1-5a01-4beb-a659-1ac75ea3e768"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:44:01.840390 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:01.840365    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c570a4d1-5a01-4beb-a659-1ac75ea3e768-cluster-tls-config" pod "c570a4d1-5a01-4beb-a659-1ac75ea3e768" (UID: "c570a4d1-5a01-4beb-a659-1ac75ea3e768"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:44:01.846752 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:01.846727    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c570a4d1-5a01-4beb-a659-1ac75ea3e768-web-config" pod "c570a4d1-5a01-4beb-a659-1ac75ea3e768" (UID: "c570a4d1-5a01-4beb-a659-1ac75ea3e768"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:44:01.933155 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:01.933122    4473 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/c570a4d1-5a01-4beb-a659-1ac75ea3e768-config-volume\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:44:01.933310 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:01.933171    4473 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c570a4d1-5a01-4beb-a659-1ac75ea3e768-web-config\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:44:01.933310 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:01.933182    4473 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/c570a4d1-5a01-4beb-a659-1ac75ea3e768-cluster-tls-config\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:44:01.933310 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:01.933193    4473 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bz72j\" (UniqueName: \"kubernetes.io/projected/c570a4d1-5a01-4beb-a659-1ac75ea3e768-kube-api-access-bz72j\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:44:01.933310 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:01.933209    4473 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/c570a4d1-5a01-4beb-a659-1ac75ea3e768-secret-alertmanager-main-tls\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:44:01.933310 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:01.933222    4473 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c570a4d1-5a01-4beb-a659-1ac75ea3e768-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:44:01.933310 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:01.933232    4473 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c570a4d1-5a01-4beb-a659-1ac75ea3e768-tls-assets\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:44:01.933310 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:01.933241    4473 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c570a4d1-5a01-4beb-a659-1ac75ea3e768-config-out\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:44:01.933310 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:01.933250    4473 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/c570a4d1-5a01-4beb-a659-1ac75ea3e768-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:44:01.933310 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:01.933260    4473 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c570a4d1-5a01-4beb-a659-1ac75ea3e768-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:44:01.933310 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:01.933268    4473 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c570a4d1-5a01-4beb-a659-1ac75ea3e768-metrics-client-ca\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:44:01.933310 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:01.933276    4473 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c570a4d1-5a01-4beb-a659-1ac75ea3e768-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:44:02.754283 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:02.754250    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:44:02.757574 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:02.757548    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-insights/periodic-gathering-bcmkf-fm7wg"]
Oct 30 21:44:02.757743 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:02.757709    4473 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-insights/periodic-gathering-bcmkf-fm7wg" podUID="bddede0e-ee07-4344-878d-10b967435e13" containerName="insights-gathering" containerID="cri-o://652149a047dc3500bf129446aecba9d8215abcd5ed7fecbe2d568df3d202f4c5" gracePeriod=30
Oct 30 21:44:02.973718 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:02.973687    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Oct 30 21:44:02.999102 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:02.999074    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-bcmkf-fm7wg"
Oct 30 21:44:03.024698 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:03.024627    4473 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Oct 30 21:44:03.025324 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:03.025304    4473 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Oct 30 21:44:03.025591 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:03.025576    4473 cpu_manager.go:414] "containerMap: removing container" podUID="c570a4d1-5a01-4beb-a659-1ac75ea3e768" containerName="config-reloader"
Oct 30 21:44:03.025669 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:03.025594    4473 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c570a4d1-5a01-4beb-a659-1ac75ea3e768" containerName="config-reloader"
Oct 30 21:44:03.025669 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:03.025609    4473 cpu_manager.go:414] "containerMap: removing container" podUID="c570a4d1-5a01-4beb-a659-1ac75ea3e768" containerName="init-config-reloader"
Oct 30 21:44:03.025669 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:03.025616    4473 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c570a4d1-5a01-4beb-a659-1ac75ea3e768" containerName="init-config-reloader"
Oct 30 21:44:03.025669 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:03.025628    4473 cpu_manager.go:414] "containerMap: removing container" podUID="c570a4d1-5a01-4beb-a659-1ac75ea3e768" containerName="alertmanager"
Oct 30 21:44:03.025669 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:03.025635    4473 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c570a4d1-5a01-4beb-a659-1ac75ea3e768" containerName="alertmanager"
Oct 30 21:44:03.025669 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:03.025663    4473 cpu_manager.go:414] "containerMap: removing container" podUID="c570a4d1-5a01-4beb-a659-1ac75ea3e768" containerName="kube-rbac-proxy-metric"
Oct 30 21:44:03.025979 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:03.025670    4473 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c570a4d1-5a01-4beb-a659-1ac75ea3e768" containerName="kube-rbac-proxy-metric"
Oct 30 21:44:03.025979 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:03.025689    4473 cpu_manager.go:414] "containerMap: removing container" podUID="c570a4d1-5a01-4beb-a659-1ac75ea3e768" containerName="kube-rbac-proxy"
Oct 30 21:44:03.025979 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:03.025698    4473 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c570a4d1-5a01-4beb-a659-1ac75ea3e768" containerName="kube-rbac-proxy"
Oct 30 21:44:03.025979 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:03.025707    4473 cpu_manager.go:414] "containerMap: removing container" podUID="bddede0e-ee07-4344-878d-10b967435e13" containerName="insights-gathering"
Oct 30 21:44:03.025979 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:03.025714    4473 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bddede0e-ee07-4344-878d-10b967435e13" containerName="insights-gathering"
Oct 30 21:44:03.025979 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:03.025725    4473 cpu_manager.go:414] "containerMap: removing container" podUID="c570a4d1-5a01-4beb-a659-1ac75ea3e768" containerName="kube-rbac-proxy-web"
Oct 30 21:44:03.025979 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:03.025731    4473 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c570a4d1-5a01-4beb-a659-1ac75ea3e768" containerName="kube-rbac-proxy-web"
Oct 30 21:44:03.025979 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:03.025746    4473 cpu_manager.go:414] "containerMap: removing container" podUID="c570a4d1-5a01-4beb-a659-1ac75ea3e768" containerName="prom-label-proxy"
Oct 30 21:44:03.025979 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:03.025754    4473 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c570a4d1-5a01-4beb-a659-1ac75ea3e768" containerName="prom-label-proxy"
Oct 30 21:44:03.025979 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:03.025811    4473 memory_manager.go:365] "RemoveStaleState removing state" podUID="c570a4d1-5a01-4beb-a659-1ac75ea3e768" containerName="kube-rbac-proxy-metric"
Oct 30 21:44:03.025979 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:03.025824    4473 memory_manager.go:365] "RemoveStaleState removing state" podUID="c570a4d1-5a01-4beb-a659-1ac75ea3e768" containerName="kube-rbac-proxy"
Oct 30 21:44:03.025979 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:03.025832    4473 memory_manager.go:365] "RemoveStaleState removing state" podUID="bddede0e-ee07-4344-878d-10b967435e13" containerName="insights-gathering"
Oct 30 21:44:03.025979 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:03.025840    4473 memory_manager.go:365] "RemoveStaleState removing state" podUID="c570a4d1-5a01-4beb-a659-1ac75ea3e768" containerName="kube-rbac-proxy-web"
Oct 30 21:44:03.025979 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:03.025850    4473 memory_manager.go:365] "RemoveStaleState removing state" podUID="c570a4d1-5a01-4beb-a659-1ac75ea3e768" containerName="prom-label-proxy"
Oct 30 21:44:03.025979 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:03.025858    4473 memory_manager.go:365] "RemoveStaleState removing state" podUID="c570a4d1-5a01-4beb-a659-1ac75ea3e768" containerName="config-reloader"
Oct 30 21:44:03.025979 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:03.025866    4473 memory_manager.go:365] "RemoveStaleState removing state" podUID="c570a4d1-5a01-4beb-a659-1ac75ea3e768" containerName="alertmanager"
Oct 30 21:44:03.028263 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:03.028245    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:44:03.084026 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:03.083998    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Oct 30 21:44:03.084307 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:03.084278    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Oct 30 21:44:03.084433 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:03.084328    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Oct 30 21:44:03.084433 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:03.084331    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Oct 30 21:44:03.084433 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:03.084386    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Oct 30 21:44:03.084433 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:03.084395    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Oct 30 21:44:03.084606 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:03.084482    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-xpr7b\""
Oct 30 21:44:03.084606 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:03.084516    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Oct 30 21:44:03.084606 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:03.084510    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Oct 30 21:44:03.134801 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:03.134762    4473 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Oct 30 21:44:03.143804 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:03.143561    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bddede0e-ee07-4344-878d-10b967435e13-kube-api-access-lnn4t\" (UniqueName: \"kubernetes.io/projected/bddede0e-ee07-4344-878d-10b967435e13-kube-api-access-lnn4t\") pod \"bddede0e-ee07-4344-878d-10b967435e13\" (UID: \"bddede0e-ee07-4344-878d-10b967435e13\") "
Oct 30 21:44:03.143804 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:03.143612    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/bddede0e-ee07-4344-878d-10b967435e13-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bddede0e-ee07-4344-878d-10b967435e13-trusted-ca-bundle\") pod \"bddede0e-ee07-4344-878d-10b967435e13\" (UID: \"bddede0e-ee07-4344-878d-10b967435e13\") "
Oct 30 21:44:03.143804 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:03.143693    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/bddede0e-ee07-4344-878d-10b967435e13-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bddede0e-ee07-4344-878d-10b967435e13-service-ca-bundle\") pod \"bddede0e-ee07-4344-878d-10b967435e13\" (UID: \"bddede0e-ee07-4344-878d-10b967435e13\") "
Oct 30 21:44:03.143804 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:03.143725    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bddede0e-ee07-4344-878d-10b967435e13-archives-path\" (UniqueName: \"kubernetes.io/empty-dir/bddede0e-ee07-4344-878d-10b967435e13-archives-path\") pod \"bddede0e-ee07-4344-878d-10b967435e13\" (UID: \"bddede0e-ee07-4344-878d-10b967435e13\") "
Oct 30 21:44:03.143804 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:03.143801    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Oct 30 21:44:03.144161 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:03.143890    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f3a5f77e-d987-4fec-90f4-88ca2f281d0e-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"f3a5f77e-d987-4fec-90f4-88ca2f281d0e\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:44:03.144161 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:03.143962    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f3a5f77e-d987-4fec-90f4-88ca2f281d0e-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"f3a5f77e-d987-4fec-90f4-88ca2f281d0e\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:44:03.144161 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:03.144031    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f3a5f77e-d987-4fec-90f4-88ca2f281d0e-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"f3a5f77e-d987-4fec-90f4-88ca2f281d0e\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:44:03.144161 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:03.144058    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/f3a5f77e-d987-4fec-90f4-88ca2f281d0e-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"f3a5f77e-d987-4fec-90f4-88ca2f281d0e\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:44:03.144161 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:03.144119    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/f3a5f77e-d987-4fec-90f4-88ca2f281d0e-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"f3a5f77e-d987-4fec-90f4-88ca2f281d0e\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:44:03.144400 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:03.144225    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f3a5f77e-d987-4fec-90f4-88ca2f281d0e-tls-assets\") pod \"alertmanager-main-0\" (UID: \"f3a5f77e-d987-4fec-90f4-88ca2f281d0e\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:44:03.144400 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:03.144256    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f3a5f77e-d987-4fec-90f4-88ca2f281d0e-config-out\") pod \"alertmanager-main-0\" (UID: \"f3a5f77e-d987-4fec-90f4-88ca2f281d0e\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:44:03.144400 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:03.144276    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bddede0e-ee07-4344-878d-10b967435e13-trusted-ca-bundle" pod "bddede0e-ee07-4344-878d-10b967435e13" (UID: "bddede0e-ee07-4344-878d-10b967435e13"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:44:03.144400 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:03.144288    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bddede0e-ee07-4344-878d-10b967435e13-archives-path" pod "bddede0e-ee07-4344-878d-10b967435e13" (UID: "bddede0e-ee07-4344-878d-10b967435e13"). InnerVolumeSpecName "archives-path". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:44:03.144400 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:03.144295    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bddede0e-ee07-4344-878d-10b967435e13-service-ca-bundle" pod "bddede0e-ee07-4344-878d-10b967435e13" (UID: "bddede0e-ee07-4344-878d-10b967435e13"). InnerVolumeSpecName "service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:44:03.144400 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:03.144295    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f3a5f77e-d987-4fec-90f4-88ca2f281d0e-web-config\") pod \"alertmanager-main-0\" (UID: \"f3a5f77e-d987-4fec-90f4-88ca2f281d0e\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:44:03.144400 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:03.144382    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/f3a5f77e-d987-4fec-90f4-88ca2f281d0e-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"f3a5f77e-d987-4fec-90f4-88ca2f281d0e\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:44:03.144400 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:03.144403    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/f3a5f77e-d987-4fec-90f4-88ca2f281d0e-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"f3a5f77e-d987-4fec-90f4-88ca2f281d0e\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:44:03.144683 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:03.144419    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f3a5f77e-d987-4fec-90f4-88ca2f281d0e-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"f3a5f77e-d987-4fec-90f4-88ca2f281d0e\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:44:03.144683 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:03.144545    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/f3a5f77e-d987-4fec-90f4-88ca2f281d0e-config-volume\") pod \"alertmanager-main-0\" (UID: \"f3a5f77e-d987-4fec-90f4-88ca2f281d0e\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:44:03.144683 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:03.144582    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qwwsk\" (UniqueName: \"kubernetes.io/projected/f3a5f77e-d987-4fec-90f4-88ca2f281d0e-kube-api-access-qwwsk\") pod \"alertmanager-main-0\" (UID: \"f3a5f77e-d987-4fec-90f4-88ca2f281d0e\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:44:03.144683 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:03.144672    4473 reconciler_common.go:299] "Volume detached for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bddede0e-ee07-4344-878d-10b967435e13-service-ca-bundle\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:44:03.144853 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:03.144692    4473 reconciler_common.go:299] "Volume detached for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/bddede0e-ee07-4344-878d-10b967435e13-archives-path\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:44:03.144853 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:03.144709    4473 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bddede0e-ee07-4344-878d-10b967435e13-trusted-ca-bundle\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:44:03.146768 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:03.146742    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bddede0e-ee07-4344-878d-10b967435e13-kube-api-access-lnn4t" pod "bddede0e-ee07-4344-878d-10b967435e13" (UID: "bddede0e-ee07-4344-878d-10b967435e13"). InnerVolumeSpecName "kube-api-access-lnn4t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:44:03.189661 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:03.189629    4473 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/periodic-gathering-9lsx9-bdv84"]
Oct 30 21:44:03.191656 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:03.191640    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-9lsx9-bdv84"
Oct 30 21:44:03.245832 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:03.245800    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/f3a5f77e-d987-4fec-90f4-88ca2f281d0e-config-volume\") pod \"alertmanager-main-0\" (UID: \"f3a5f77e-d987-4fec-90f4-88ca2f281d0e\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:44:03.246058 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:03.245840    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qwwsk\" (UniqueName: \"kubernetes.io/projected/f3a5f77e-d987-4fec-90f4-88ca2f281d0e-kube-api-access-qwwsk\") pod \"alertmanager-main-0\" (UID: \"f3a5f77e-d987-4fec-90f4-88ca2f281d0e\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:44:03.246058 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:03.245864    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f3a5f77e-d987-4fec-90f4-88ca2f281d0e-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"f3a5f77e-d987-4fec-90f4-88ca2f281d0e\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:44:03.246058 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:03.245888    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f3a5f77e-d987-4fec-90f4-88ca2f281d0e-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"f3a5f77e-d987-4fec-90f4-88ca2f281d0e\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:44:03.246058 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:03.245937    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f3a5f77e-d987-4fec-90f4-88ca2f281d0e-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"f3a5f77e-d987-4fec-90f4-88ca2f281d0e\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:44:03.246058 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:03.245981    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/f3a5f77e-d987-4fec-90f4-88ca2f281d0e-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"f3a5f77e-d987-4fec-90f4-88ca2f281d0e\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:44:03.246058 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:03.246011    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/f3a5f77e-d987-4fec-90f4-88ca2f281d0e-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"f3a5f77e-d987-4fec-90f4-88ca2f281d0e\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:44:03.246058 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:03.246053    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f3a5f77e-d987-4fec-90f4-88ca2f281d0e-tls-assets\") pod \"alertmanager-main-0\" (UID: \"f3a5f77e-d987-4fec-90f4-88ca2f281d0e\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:44:03.246423 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:03.246080    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f3a5f77e-d987-4fec-90f4-88ca2f281d0e-config-out\") pod \"alertmanager-main-0\" (UID: \"f3a5f77e-d987-4fec-90f4-88ca2f281d0e\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:44:03.246423 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:03.246111    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f3a5f77e-d987-4fec-90f4-88ca2f281d0e-web-config\") pod \"alertmanager-main-0\" (UID: \"f3a5f77e-d987-4fec-90f4-88ca2f281d0e\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:44:03.246423 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:03.246141    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/f3a5f77e-d987-4fec-90f4-88ca2f281d0e-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"f3a5f77e-d987-4fec-90f4-88ca2f281d0e\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:44:03.246423 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:03.246186    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/f3a5f77e-d987-4fec-90f4-88ca2f281d0e-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"f3a5f77e-d987-4fec-90f4-88ca2f281d0e\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:44:03.246423 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:03.246232    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f3a5f77e-d987-4fec-90f4-88ca2f281d0e-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"f3a5f77e-d987-4fec-90f4-88ca2f281d0e\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:44:03.246423 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:03.246302    4473 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lnn4t\" (UniqueName: \"kubernetes.io/projected/bddede0e-ee07-4344-878d-10b967435e13-kube-api-access-lnn4t\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:44:03.246873 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:03.246541    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/f3a5f77e-d987-4fec-90f4-88ca2f281d0e-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"f3a5f77e-d987-4fec-90f4-88ca2f281d0e\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:44:03.247312 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:03.246939    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f3a5f77e-d987-4fec-90f4-88ca2f281d0e-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"f3a5f77e-d987-4fec-90f4-88ca2f281d0e\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:44:03.247312 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:03.247054    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f3a5f77e-d987-4fec-90f4-88ca2f281d0e-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"f3a5f77e-d987-4fec-90f4-88ca2f281d0e\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:44:03.249162 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:03.248909    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f3a5f77e-d987-4fec-90f4-88ca2f281d0e-config-out\") pod \"alertmanager-main-0\" (UID: \"f3a5f77e-d987-4fec-90f4-88ca2f281d0e\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:44:03.249162 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:03.249109    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/f3a5f77e-d987-4fec-90f4-88ca2f281d0e-config-volume\") pod \"alertmanager-main-0\" (UID: \"f3a5f77e-d987-4fec-90f4-88ca2f281d0e\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:44:03.249371 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:03.249347    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f3a5f77e-d987-4fec-90f4-88ca2f281d0e-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"f3a5f77e-d987-4fec-90f4-88ca2f281d0e\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:44:03.249528 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:03.249500    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/f3a5f77e-d987-4fec-90f4-88ca2f281d0e-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"f3a5f77e-d987-4fec-90f4-88ca2f281d0e\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:44:03.249879 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:03.249857    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f3a5f77e-d987-4fec-90f4-88ca2f281d0e-tls-assets\") pod \"alertmanager-main-0\" (UID: \"f3a5f77e-d987-4fec-90f4-88ca2f281d0e\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:44:03.249955 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:03.249892    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f3a5f77e-d987-4fec-90f4-88ca2f281d0e-web-config\") pod \"alertmanager-main-0\" (UID: \"f3a5f77e-d987-4fec-90f4-88ca2f281d0e\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:44:03.250090 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:03.250069    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/f3a5f77e-d987-4fec-90f4-88ca2f281d0e-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"f3a5f77e-d987-4fec-90f4-88ca2f281d0e\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:44:03.250296 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:03.250280    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f3a5f77e-d987-4fec-90f4-88ca2f281d0e-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"f3a5f77e-d987-4fec-90f4-88ca2f281d0e\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:44:03.250675 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:03.250660    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/f3a5f77e-d987-4fec-90f4-88ca2f281d0e-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"f3a5f77e-d987-4fec-90f4-88ca2f281d0e\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:44:03.254476 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:03.254455    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/periodic-gathering-9lsx9-bdv84"]
Oct 30 21:44:03.307161 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:03.307077    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qwwsk\" (UniqueName: \"kubernetes.io/projected/f3a5f77e-d987-4fec-90f4-88ca2f281d0e-kube-api-access-qwwsk\") pod \"alertmanager-main-0\" (UID: \"f3a5f77e-d987-4fec-90f4-88ca2f281d0e\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:44:03.344363 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:03.344341    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:44:03.346602 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:03.346580    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/79ffeeb5-a231-4dd0-9db3-857cd39cc95e-trusted-ca-bundle\") pod \"periodic-gathering-9lsx9-bdv84\" (UID: \"79ffeeb5-a231-4dd0-9db3-857cd39cc95e\") " pod="openshift-insights/periodic-gathering-9lsx9-bdv84"
Oct 30 21:44:03.346669 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:03.346635    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/79ffeeb5-a231-4dd0-9db3-857cd39cc95e-archives-path\") pod \"periodic-gathering-9lsx9-bdv84\" (UID: \"79ffeeb5-a231-4dd0-9db3-857cd39cc95e\") " pod="openshift-insights/periodic-gathering-9lsx9-bdv84"
Oct 30 21:44:03.346787 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:03.346713    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bdl48\" (UniqueName: \"kubernetes.io/projected/79ffeeb5-a231-4dd0-9db3-857cd39cc95e-kube-api-access-bdl48\") pod \"periodic-gathering-9lsx9-bdv84\" (UID: \"79ffeeb5-a231-4dd0-9db3-857cd39cc95e\") " pod="openshift-insights/periodic-gathering-9lsx9-bdv84"
Oct 30 21:44:03.346820 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:03.346801    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/79ffeeb5-a231-4dd0-9db3-857cd39cc95e-service-ca-bundle\") pod \"periodic-gathering-9lsx9-bdv84\" (UID: \"79ffeeb5-a231-4dd0-9db3-857cd39cc95e\") " pod="openshift-insights/periodic-gathering-9lsx9-bdv84"
Oct 30 21:44:03.447423 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:03.447393    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bdl48\" (UniqueName: \"kubernetes.io/projected/79ffeeb5-a231-4dd0-9db3-857cd39cc95e-kube-api-access-bdl48\") pod \"periodic-gathering-9lsx9-bdv84\" (UID: \"79ffeeb5-a231-4dd0-9db3-857cd39cc95e\") " pod="openshift-insights/periodic-gathering-9lsx9-bdv84"
Oct 30 21:44:03.447423 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:03.447427    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/79ffeeb5-a231-4dd0-9db3-857cd39cc95e-service-ca-bundle\") pod \"periodic-gathering-9lsx9-bdv84\" (UID: \"79ffeeb5-a231-4dd0-9db3-857cd39cc95e\") " pod="openshift-insights/periodic-gathering-9lsx9-bdv84"
Oct 30 21:44:03.447675 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:03.447457    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/79ffeeb5-a231-4dd0-9db3-857cd39cc95e-trusted-ca-bundle\") pod \"periodic-gathering-9lsx9-bdv84\" (UID: \"79ffeeb5-a231-4dd0-9db3-857cd39cc95e\") " pod="openshift-insights/periodic-gathering-9lsx9-bdv84"
Oct 30 21:44:03.447675 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:03.447488    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/79ffeeb5-a231-4dd0-9db3-857cd39cc95e-archives-path\") pod \"periodic-gathering-9lsx9-bdv84\" (UID: \"79ffeeb5-a231-4dd0-9db3-857cd39cc95e\") " pod="openshift-insights/periodic-gathering-9lsx9-bdv84"
Oct 30 21:44:03.447798 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:03.447784    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/79ffeeb5-a231-4dd0-9db3-857cd39cc95e-archives-path\") pod \"periodic-gathering-9lsx9-bdv84\" (UID: \"79ffeeb5-a231-4dd0-9db3-857cd39cc95e\") " pod="openshift-insights/periodic-gathering-9lsx9-bdv84"
Oct 30 21:44:03.448100 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:03.448082    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/79ffeeb5-a231-4dd0-9db3-857cd39cc95e-service-ca-bundle\") pod \"periodic-gathering-9lsx9-bdv84\" (UID: \"79ffeeb5-a231-4dd0-9db3-857cd39cc95e\") " pod="openshift-insights/periodic-gathering-9lsx9-bdv84"
Oct 30 21:44:03.448232 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:03.448215    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/79ffeeb5-a231-4dd0-9db3-857cd39cc95e-trusted-ca-bundle\") pod \"periodic-gathering-9lsx9-bdv84\" (UID: \"79ffeeb5-a231-4dd0-9db3-857cd39cc95e\") " pod="openshift-insights/periodic-gathering-9lsx9-bdv84"
Oct 30 21:44:03.511500 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:03.511470    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bdl48\" (UniqueName: \"kubernetes.io/projected/79ffeeb5-a231-4dd0-9db3-857cd39cc95e-kube-api-access-bdl48\") pod \"periodic-gathering-9lsx9-bdv84\" (UID: \"79ffeeb5-a231-4dd0-9db3-857cd39cc95e\") " pod="openshift-insights/periodic-gathering-9lsx9-bdv84"
Oct 30 21:44:03.759542 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:03.759506    4473 generic.go:356] "Generic (PLEG): container finished" podID="bddede0e-ee07-4344-878d-10b967435e13" containerID="652149a047dc3500bf129446aecba9d8215abcd5ed7fecbe2d568df3d202f4c5" exitCode=2
Oct 30 21:44:03.760025 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:03.759562    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-bcmkf-fm7wg" event={"ID":"bddede0e-ee07-4344-878d-10b967435e13","Type":"ContainerDied","Data":"652149a047dc3500bf129446aecba9d8215abcd5ed7fecbe2d568df3d202f4c5"}
Oct 30 21:44:03.760025 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:03.759577    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-bcmkf-fm7wg"
Oct 30 21:44:03.760025 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:03.759626    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-bcmkf-fm7wg" event={"ID":"bddede0e-ee07-4344-878d-10b967435e13","Type":"ContainerDied","Data":"76411d00e5efeac0f8e2cdc0c51751cb906f91e91e8b427c7cd78c85f25ebfa6"}
Oct 30 21:44:03.760025 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:03.759644    4473 scope.go:122] "RemoveContainer" containerID="652149a047dc3500bf129446aecba9d8215abcd5ed7fecbe2d568df3d202f4c5"
Oct 30 21:44:03.767557 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:03.767538    4473 scope.go:122] "RemoveContainer" containerID="652149a047dc3500bf129446aecba9d8215abcd5ed7fecbe2d568df3d202f4c5"
Oct 30 21:44:03.767825 ip-10-0-140-18 kubenswrapper[4473]: E1030 21:44:03.767804    4473 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"652149a047dc3500bf129446aecba9d8215abcd5ed7fecbe2d568df3d202f4c5\": container with ID starting with 652149a047dc3500bf129446aecba9d8215abcd5ed7fecbe2d568df3d202f4c5 not found: ID does not exist" containerID="652149a047dc3500bf129446aecba9d8215abcd5ed7fecbe2d568df3d202f4c5"
Oct 30 21:44:03.767898 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:03.767834    4473 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"652149a047dc3500bf129446aecba9d8215abcd5ed7fecbe2d568df3d202f4c5"} err="failed to get container status \"652149a047dc3500bf129446aecba9d8215abcd5ed7fecbe2d568df3d202f4c5\": rpc error: code = NotFound desc = could not find container \"652149a047dc3500bf129446aecba9d8215abcd5ed7fecbe2d568df3d202f4c5\": container with ID starting with 652149a047dc3500bf129446aecba9d8215abcd5ed7fecbe2d568df3d202f4c5 not found: ID does not exist"
Oct 30 21:44:03.775684 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:03.775662    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Oct 30 21:44:03.777755 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:44:03.777734    4473 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf3a5f77e_d987_4fec_90f4_88ca2f281d0e.slice/crio-682b690a752482c7c09b96808895d16a5da85992424b627fc6c54f1a393580e3 WatchSource:0}: Error finding container 682b690a752482c7c09b96808895d16a5da85992424b627fc6c54f1a393580e3: Status 404 returned error can't find the container with id 682b690a752482c7c09b96808895d16a5da85992424b627fc6c54f1a393580e3
Oct 30 21:44:03.801540 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:03.801519    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-9lsx9-bdv84"
Oct 30 21:44:03.927327 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:03.927294    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-insights/periodic-gathering-bcmkf-fm7wg"]
Oct 30 21:44:03.931502 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:03.931479    4473 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-insights/periodic-gathering-bcmkf-fm7wg"]
Oct 30 21:44:04.173669 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:04.173648    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/periodic-gathering-9lsx9-bdv84"]
Oct 30 21:44:04.175608 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:44:04.175583    4473 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79ffeeb5_a231_4dd0_9db3_857cd39cc95e.slice/crio-7f86c3320f80ee79bc5f014338c9ddcd2dee3e1094550b4a6e87b8f886d3dfee WatchSource:0}: Error finding container 7f86c3320f80ee79bc5f014338c9ddcd2dee3e1094550b4a6e87b8f886d3dfee: Status 404 returned error can't find the container with id 7f86c3320f80ee79bc5f014338c9ddcd2dee3e1094550b4a6e87b8f886d3dfee
Oct 30 21:44:04.354383 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:04.354349    4473 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="bddede0e-ee07-4344-878d-10b967435e13" path="/var/lib/kubelet/pods/bddede0e-ee07-4344-878d-10b967435e13/volumes"
Oct 30 21:44:04.354747 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:04.354735    4473 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c570a4d1-5a01-4beb-a659-1ac75ea3e768" path="/var/lib/kubelet/pods/c570a4d1-5a01-4beb-a659-1ac75ea3e768/volumes"
Oct 30 21:44:04.764603 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:04.764573    4473 generic.go:356] "Generic (PLEG): container finished" podID="f3a5f77e-d987-4fec-90f4-88ca2f281d0e" containerID="4e5fbd6edec0fb84896303c801a6b46c89171c56b0e3986569a5c60fb5c419de" exitCode=0
Oct 30 21:44:04.765086 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:04.764650    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f3a5f77e-d987-4fec-90f4-88ca2f281d0e","Type":"ContainerDied","Data":"4e5fbd6edec0fb84896303c801a6b46c89171c56b0e3986569a5c60fb5c419de"}
Oct 30 21:44:04.765086 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:04.764729    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f3a5f77e-d987-4fec-90f4-88ca2f281d0e","Type":"ContainerStarted","Data":"682b690a752482c7c09b96808895d16a5da85992424b627fc6c54f1a393580e3"}
Oct 30 21:44:04.767100 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:04.767070    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-9lsx9-bdv84" event={"ID":"79ffeeb5-a231-4dd0-9db3-857cd39cc95e","Type":"ContainerStarted","Data":"73f9f86117ce6e11b709ee75476293911e6882b07de1f2b28fc6fdb4d8a846b7"}
Oct 30 21:44:04.767201 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:04.767115    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-9lsx9-bdv84" event={"ID":"79ffeeb5-a231-4dd0-9db3-857cd39cc95e","Type":"ContainerStarted","Data":"7f86c3320f80ee79bc5f014338c9ddcd2dee3e1094550b4a6e87b8f886d3dfee"}
Oct 30 21:44:05.011025 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:05.010965    4473 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-insights/periodic-gathering-9lsx9-bdv84" podStartSLOduration=2.010946535 podStartE2EDuration="2.010946535s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:44:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:44:05.010019002 +0000 UTC m=+90.783387489" watchObservedRunningTime="2026-10-30 21:44:05.010946535 +0000 UTC m=+90.784315017"
Oct 30 21:44:05.774240 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:05.774206    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f3a5f77e-d987-4fec-90f4-88ca2f281d0e","Type":"ContainerStarted","Data":"95106734ea7f9d77ca1bb0068395ad1bc2221284daefd105cb65bf757322f0bd"}
Oct 30 21:44:05.774611 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:05.774298    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f3a5f77e-d987-4fec-90f4-88ca2f281d0e","Type":"ContainerStarted","Data":"282baf9d62fd76b479093059067de6252055813bc2b9fbeda5327baf9222e8ea"}
Oct 30 21:44:05.774611 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:05.774329    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f3a5f77e-d987-4fec-90f4-88ca2f281d0e","Type":"ContainerStarted","Data":"cc87dce3823f0d8c52856b29f01aac53322b760397824e41482592463ac5d6c6"}
Oct 30 21:44:05.774611 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:05.774361    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f3a5f77e-d987-4fec-90f4-88ca2f281d0e","Type":"ContainerStarted","Data":"b98d2936114f3f8b363ac5b2d5e2f436252984d8ffaa27f3d35741d8fa85ea97"}
Oct 30 21:44:05.774611 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:05.774397    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f3a5f77e-d987-4fec-90f4-88ca2f281d0e","Type":"ContainerStarted","Data":"eb8fa5c21bd08132da6a0082a76f07a9e3f863a12b9115884a7c301b22f65bcf"}
Oct 30 21:44:05.774611 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:05.774434    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f3a5f77e-d987-4fec-90f4-88ca2f281d0e","Type":"ContainerStarted","Data":"7bcda9d019c1fdcc8fba63a84810780658e84520a3780481220fe0526c1fde43"}
Oct 30 21:44:05.900726 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:05.900674    4473 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.8812569139999997 podStartE2EDuration="3.900660929s" totalImagesPullingTime="0s" totalInitContainerRuntime="19.404015ms" isStatefulPod=false podCreationTimestamp="2026-10-30 21:44:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:44:05.899996887 +0000 UTC m=+91.673365374" watchObservedRunningTime="2026-10-30 21:44:05.900660929 +0000 UTC m=+91.674029414"
Oct 30 21:44:06.868753 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:06.868714    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7c5745455d-z9bjx"
Oct 30 21:44:06.869144 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:06.868777    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7c5745455d-z9bjx"
Oct 30 21:44:06.873299 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:06.873278    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7c5745455d-z9bjx"
Oct 30 21:44:07.787358 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:07.787325    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7c5745455d-z9bjx"
Oct 30 21:44:07.949723 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:07.949686    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-548c99d9-nf7hj"]
Oct 30 21:44:20.322787 ip-10-0-140-18 kubenswrapper[4473]: E1030 21:44:20.322742    4473 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fncvg" containerName="extractor"
Oct 30 21:44:32.970323 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:32.970259    4473 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-console/console-548c99d9-nf7hj" podUID="e53bbdd4-5b5b-4a9d-bd11-8796b18af1b3" containerName="console" containerID="cri-o://7402d5acb284aba4de6964ccc4ca9c52b23dbca9b3f7f10f965f2a05637fa79e" gracePeriod=15
Oct 30 21:44:33.218868 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:33.218844    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-548c99d9-nf7hj"
Oct 30 21:44:33.298111 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:33.298014    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e53bbdd4-5b5b-4a9d-bd11-8796b18af1b3-console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e53bbdd4-5b5b-4a9d-bd11-8796b18af1b3-console-serving-cert\") pod \"e53bbdd4-5b5b-4a9d-bd11-8796b18af1b3\" (UID: \"e53bbdd4-5b5b-4a9d-bd11-8796b18af1b3\") "
Oct 30 21:44:33.298111 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:33.298073    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e53bbdd4-5b5b-4a9d-bd11-8796b18af1b3-service-ca\" (UniqueName: \"kubernetes.io/configmap/e53bbdd4-5b5b-4a9d-bd11-8796b18af1b3-service-ca\") pod \"e53bbdd4-5b5b-4a9d-bd11-8796b18af1b3\" (UID: \"e53bbdd4-5b5b-4a9d-bd11-8796b18af1b3\") "
Oct 30 21:44:33.298111 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:33.298103    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e53bbdd4-5b5b-4a9d-bd11-8796b18af1b3-console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e53bbdd4-5b5b-4a9d-bd11-8796b18af1b3-console-oauth-config\") pod \"e53bbdd4-5b5b-4a9d-bd11-8796b18af1b3\" (UID: \"e53bbdd4-5b5b-4a9d-bd11-8796b18af1b3\") "
Oct 30 21:44:33.298371 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:33.298140    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e53bbdd4-5b5b-4a9d-bd11-8796b18af1b3-kube-api-access-9tt4n\" (UniqueName: \"kubernetes.io/projected/e53bbdd4-5b5b-4a9d-bd11-8796b18af1b3-kube-api-access-9tt4n\") pod \"e53bbdd4-5b5b-4a9d-bd11-8796b18af1b3\" (UID: \"e53bbdd4-5b5b-4a9d-bd11-8796b18af1b3\") "
Oct 30 21:44:33.298371 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:33.298156    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e53bbdd4-5b5b-4a9d-bd11-8796b18af1b3-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e53bbdd4-5b5b-4a9d-bd11-8796b18af1b3-trusted-ca-bundle\") pod \"e53bbdd4-5b5b-4a9d-bd11-8796b18af1b3\" (UID: \"e53bbdd4-5b5b-4a9d-bd11-8796b18af1b3\") "
Oct 30 21:44:33.298371 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:33.298179    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e53bbdd4-5b5b-4a9d-bd11-8796b18af1b3-console-config\" (UniqueName: \"kubernetes.io/configmap/e53bbdd4-5b5b-4a9d-bd11-8796b18af1b3-console-config\") pod \"e53bbdd4-5b5b-4a9d-bd11-8796b18af1b3\" (UID: \"e53bbdd4-5b5b-4a9d-bd11-8796b18af1b3\") "
Oct 30 21:44:33.298371 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:33.298223    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e53bbdd4-5b5b-4a9d-bd11-8796b18af1b3-oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e53bbdd4-5b5b-4a9d-bd11-8796b18af1b3-oauth-serving-cert\") pod \"e53bbdd4-5b5b-4a9d-bd11-8796b18af1b3\" (UID: \"e53bbdd4-5b5b-4a9d-bd11-8796b18af1b3\") "
Oct 30 21:44:33.298371 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:33.298254    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e53bbdd4-5b5b-4a9d-bd11-8796b18af1b3-tmp\" (UniqueName: \"kubernetes.io/empty-dir/e53bbdd4-5b5b-4a9d-bd11-8796b18af1b3-tmp\") pod \"e53bbdd4-5b5b-4a9d-bd11-8796b18af1b3\" (UID: \"e53bbdd4-5b5b-4a9d-bd11-8796b18af1b3\") "
Oct 30 21:44:33.298628 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:33.298524    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e53bbdd4-5b5b-4a9d-bd11-8796b18af1b3-service-ca" pod "e53bbdd4-5b5b-4a9d-bd11-8796b18af1b3" (UID: "e53bbdd4-5b5b-4a9d-bd11-8796b18af1b3"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:44:33.298685 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:33.298667    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e53bbdd4-5b5b-4a9d-bd11-8796b18af1b3-tmp" pod "e53bbdd4-5b5b-4a9d-bd11-8796b18af1b3" (UID: "e53bbdd4-5b5b-4a9d-bd11-8796b18af1b3"). InnerVolumeSpecName "tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:44:33.298734 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:33.298680    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e53bbdd4-5b5b-4a9d-bd11-8796b18af1b3-trusted-ca-bundle" pod "e53bbdd4-5b5b-4a9d-bd11-8796b18af1b3" (UID: "e53bbdd4-5b5b-4a9d-bd11-8796b18af1b3"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:44:33.299013 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:33.298798    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e53bbdd4-5b5b-4a9d-bd11-8796b18af1b3-console-config" pod "e53bbdd4-5b5b-4a9d-bd11-8796b18af1b3" (UID: "e53bbdd4-5b5b-4a9d-bd11-8796b18af1b3"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:44:33.299144 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:33.299059    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e53bbdd4-5b5b-4a9d-bd11-8796b18af1b3-oauth-serving-cert" pod "e53bbdd4-5b5b-4a9d-bd11-8796b18af1b3" (UID: "e53bbdd4-5b5b-4a9d-bd11-8796b18af1b3"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:44:33.300369 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:33.300347    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e53bbdd4-5b5b-4a9d-bd11-8796b18af1b3-console-serving-cert" pod "e53bbdd4-5b5b-4a9d-bd11-8796b18af1b3" (UID: "e53bbdd4-5b5b-4a9d-bd11-8796b18af1b3"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:44:33.300483 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:33.300467    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e53bbdd4-5b5b-4a9d-bd11-8796b18af1b3-kube-api-access-9tt4n" pod "e53bbdd4-5b5b-4a9d-bd11-8796b18af1b3" (UID: "e53bbdd4-5b5b-4a9d-bd11-8796b18af1b3"). InnerVolumeSpecName "kube-api-access-9tt4n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:44:33.300524 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:33.300473    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e53bbdd4-5b5b-4a9d-bd11-8796b18af1b3-console-oauth-config" pod "e53bbdd4-5b5b-4a9d-bd11-8796b18af1b3" (UID: "e53bbdd4-5b5b-4a9d-bd11-8796b18af1b3"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:44:33.399583 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:33.399547    4473 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e53bbdd4-5b5b-4a9d-bd11-8796b18af1b3-console-config\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:44:33.399583 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:33.399576    4473 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e53bbdd4-5b5b-4a9d-bd11-8796b18af1b3-oauth-serving-cert\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:44:33.399583 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:33.399585    4473 reconciler_common.go:299] "Volume detached for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e53bbdd4-5b5b-4a9d-bd11-8796b18af1b3-tmp\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:44:33.399583 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:33.399594    4473 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e53bbdd4-5b5b-4a9d-bd11-8796b18af1b3-console-serving-cert\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:44:33.399845 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:33.399604    4473 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e53bbdd4-5b5b-4a9d-bd11-8796b18af1b3-service-ca\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:44:33.399845 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:33.399612    4473 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e53bbdd4-5b5b-4a9d-bd11-8796b18af1b3-console-oauth-config\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:44:33.399845 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:33.399621    4473 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9tt4n\" (UniqueName: \"kubernetes.io/projected/e53bbdd4-5b5b-4a9d-bd11-8796b18af1b3-kube-api-access-9tt4n\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:44:33.399845 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:33.399631    4473 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e53bbdd4-5b5b-4a9d-bd11-8796b18af1b3-trusted-ca-bundle\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:44:33.890536 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:33.890497    4473 generic.go:356] "Generic (PLEG): container finished" podID="e53bbdd4-5b5b-4a9d-bd11-8796b18af1b3" containerID="7402d5acb284aba4de6964ccc4ca9c52b23dbca9b3f7f10f965f2a05637fa79e" exitCode=0
Oct 30 21:44:33.890702 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:33.890546    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-548c99d9-nf7hj"
Oct 30 21:44:33.890702 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:33.890570    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-548c99d9-nf7hj" event={"ID":"e53bbdd4-5b5b-4a9d-bd11-8796b18af1b3","Type":"ContainerDied","Data":"7402d5acb284aba4de6964ccc4ca9c52b23dbca9b3f7f10f965f2a05637fa79e"}
Oct 30 21:44:33.890702 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:33.890618    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-548c99d9-nf7hj" event={"ID":"e53bbdd4-5b5b-4a9d-bd11-8796b18af1b3","Type":"ContainerDied","Data":"3a4e0f570161c1182a2f83ce69dea704b3e2ff2ed0da6e0a2da020c8c4f26bf5"}
Oct 30 21:44:33.890702 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:33.890643    4473 scope.go:122] "RemoveContainer" containerID="7402d5acb284aba4de6964ccc4ca9c52b23dbca9b3f7f10f965f2a05637fa79e"
Oct 30 21:44:33.899060 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:33.899039    4473 scope.go:122] "RemoveContainer" containerID="7402d5acb284aba4de6964ccc4ca9c52b23dbca9b3f7f10f965f2a05637fa79e"
Oct 30 21:44:33.899325 ip-10-0-140-18 kubenswrapper[4473]: E1030 21:44:33.899305    4473 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7402d5acb284aba4de6964ccc4ca9c52b23dbca9b3f7f10f965f2a05637fa79e\": container with ID starting with 7402d5acb284aba4de6964ccc4ca9c52b23dbca9b3f7f10f965f2a05637fa79e not found: ID does not exist" containerID="7402d5acb284aba4de6964ccc4ca9c52b23dbca9b3f7f10f965f2a05637fa79e"
Oct 30 21:44:33.899367 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:33.899335    4473 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7402d5acb284aba4de6964ccc4ca9c52b23dbca9b3f7f10f965f2a05637fa79e"} err="failed to get container status \"7402d5acb284aba4de6964ccc4ca9c52b23dbca9b3f7f10f965f2a05637fa79e\": rpc error: code = NotFound desc = could not find container \"7402d5acb284aba4de6964ccc4ca9c52b23dbca9b3f7f10f965f2a05637fa79e\": container with ID starting with 7402d5acb284aba4de6964ccc4ca9c52b23dbca9b3f7f10f965f2a05637fa79e not found: ID does not exist"
Oct 30 21:44:34.057561 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:34.057522    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-548c99d9-nf7hj"]
Oct 30 21:44:34.106848 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:34.106815    4473 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-548c99d9-nf7hj"]
Oct 30 21:44:34.353945 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:44:34.353896    4473 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e53bbdd4-5b5b-4a9d-bd11-8796b18af1b3" path="/var/lib/kubelet/pods/e53bbdd4-5b5b-4a9d-bd11-8796b18af1b3/volumes"
Oct 30 21:44:50.321579 ip-10-0-140-18 kubenswrapper[4473]: E1030 21:44:50.321494    4473 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fncvg" containerName="extractor"
Oct 30 21:45:20.323472 ip-10-0-140-18 kubenswrapper[4473]: E1030 21:45:20.323426    4473 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fncvg" containerName="extractor"
Oct 30 21:45:29.092644 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:45:29.092550    4473 generic.go:356] "Generic (PLEG): container finished" podID="79ffeeb5-a231-4dd0-9db3-857cd39cc95e" containerID="73f9f86117ce6e11b709ee75476293911e6882b07de1f2b28fc6fdb4d8a846b7" exitCode=0
Oct 30 21:45:29.092644 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:45:29.092623    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-9lsx9-bdv84" event={"ID":"79ffeeb5-a231-4dd0-9db3-857cd39cc95e","Type":"ContainerDied","Data":"73f9f86117ce6e11b709ee75476293911e6882b07de1f2b28fc6fdb4d8a846b7"}
Oct 30 21:45:30.222990 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:45:30.222965    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-9lsx9-bdv84"
Oct 30 21:45:30.328026 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:45:30.327989    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/79ffeeb5-a231-4dd0-9db3-857cd39cc95e-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/79ffeeb5-a231-4dd0-9db3-857cd39cc95e-service-ca-bundle\") pod \"79ffeeb5-a231-4dd0-9db3-857cd39cc95e\" (UID: \"79ffeeb5-a231-4dd0-9db3-857cd39cc95e\") "
Oct 30 21:45:30.328218 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:45:30.328039    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/79ffeeb5-a231-4dd0-9db3-857cd39cc95e-kube-api-access-bdl48\" (UniqueName: \"kubernetes.io/projected/79ffeeb5-a231-4dd0-9db3-857cd39cc95e-kube-api-access-bdl48\") pod \"79ffeeb5-a231-4dd0-9db3-857cd39cc95e\" (UID: \"79ffeeb5-a231-4dd0-9db3-857cd39cc95e\") "
Oct 30 21:45:30.328218 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:45:30.328077    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/79ffeeb5-a231-4dd0-9db3-857cd39cc95e-archives-path\" (UniqueName: \"kubernetes.io/empty-dir/79ffeeb5-a231-4dd0-9db3-857cd39cc95e-archives-path\") pod \"79ffeeb5-a231-4dd0-9db3-857cd39cc95e\" (UID: \"79ffeeb5-a231-4dd0-9db3-857cd39cc95e\") "
Oct 30 21:45:30.328218 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:45:30.328095    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/79ffeeb5-a231-4dd0-9db3-857cd39cc95e-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/79ffeeb5-a231-4dd0-9db3-857cd39cc95e-trusted-ca-bundle\") pod \"79ffeeb5-a231-4dd0-9db3-857cd39cc95e\" (UID: \"79ffeeb5-a231-4dd0-9db3-857cd39cc95e\") "
Oct 30 21:45:30.328442 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:45:30.328413    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/79ffeeb5-a231-4dd0-9db3-857cd39cc95e-archives-path" pod "79ffeeb5-a231-4dd0-9db3-857cd39cc95e" (UID: "79ffeeb5-a231-4dd0-9db3-857cd39cc95e"). InnerVolumeSpecName "archives-path". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:45:30.328496 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:45:30.328439    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/79ffeeb5-a231-4dd0-9db3-857cd39cc95e-trusted-ca-bundle" pod "79ffeeb5-a231-4dd0-9db3-857cd39cc95e" (UID: "79ffeeb5-a231-4dd0-9db3-857cd39cc95e"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:45:30.328496 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:45:30.328450    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/79ffeeb5-a231-4dd0-9db3-857cd39cc95e-service-ca-bundle" pod "79ffeeb5-a231-4dd0-9db3-857cd39cc95e" (UID: "79ffeeb5-a231-4dd0-9db3-857cd39cc95e"). InnerVolumeSpecName "service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:45:30.330486 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:45:30.330454    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/79ffeeb5-a231-4dd0-9db3-857cd39cc95e-kube-api-access-bdl48" pod "79ffeeb5-a231-4dd0-9db3-857cd39cc95e" (UID: "79ffeeb5-a231-4dd0-9db3-857cd39cc95e"). InnerVolumeSpecName "kube-api-access-bdl48". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:45:30.429103 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:45:30.429065    4473 reconciler_common.go:299] "Volume detached for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/79ffeeb5-a231-4dd0-9db3-857cd39cc95e-service-ca-bundle\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:45:30.429103 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:45:30.429102    4473 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bdl48\" (UniqueName: \"kubernetes.io/projected/79ffeeb5-a231-4dd0-9db3-857cd39cc95e-kube-api-access-bdl48\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:45:30.429103 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:45:30.429113    4473 reconciler_common.go:299] "Volume detached for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/79ffeeb5-a231-4dd0-9db3-857cd39cc95e-archives-path\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:45:30.429103 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:45:30.429122    4473 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/79ffeeb5-a231-4dd0-9db3-857cd39cc95e-trusted-ca-bundle\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:45:31.101563 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:45:31.101518    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-9lsx9-bdv84" event={"ID":"79ffeeb5-a231-4dd0-9db3-857cd39cc95e","Type":"ContainerDied","Data":"7f86c3320f80ee79bc5f014338c9ddcd2dee3e1094550b4a6e87b8f886d3dfee"}
Oct 30 21:45:31.101563 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:45:31.101538    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-9lsx9-bdv84"
Oct 30 21:45:31.101770 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:45:31.101571    4473 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7f86c3320f80ee79bc5f014338c9ddcd2dee3e1094550b4a6e87b8f886d3dfee"
Oct 30 21:45:50.322776 ip-10-0-140-18 kubenswrapper[4473]: E1030 21:45:50.322743    4473 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fncvg" containerName="extractor"
Oct 30 21:46:20.322979 ip-10-0-140-18 kubenswrapper[4473]: E1030 21:46:20.322893    4473 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fncvg" containerName="extractor"
Oct 30 21:46:34.441285 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:46:34.441255    4473 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644k758g"]
Oct 30 21:46:34.441771 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:46:34.441576    4473 cpu_manager.go:414] "containerMap: removing container" podUID="e53bbdd4-5b5b-4a9d-bd11-8796b18af1b3" containerName="console"
Oct 30 21:46:34.441771 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:46:34.441587    4473 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e53bbdd4-5b5b-4a9d-bd11-8796b18af1b3" containerName="console"
Oct 30 21:46:34.441771 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:46:34.441595    4473 cpu_manager.go:414] "containerMap: removing container" podUID="79ffeeb5-a231-4dd0-9db3-857cd39cc95e" containerName="insights-gathering"
Oct 30 21:46:34.441771 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:46:34.441601    4473 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="79ffeeb5-a231-4dd0-9db3-857cd39cc95e" containerName="insights-gathering"
Oct 30 21:46:34.441771 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:46:34.441658    4473 memory_manager.go:365] "RemoveStaleState removing state" podUID="e53bbdd4-5b5b-4a9d-bd11-8796b18af1b3" containerName="console"
Oct 30 21:46:34.441771 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:46:34.441667    4473 memory_manager.go:365] "RemoveStaleState removing state" podUID="79ffeeb5-a231-4dd0-9db3-857cd39cc95e" containerName="insights-gathering"
Oct 30 21:46:34.444635 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:46:34.444619    4473 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651czllrv"]
Oct 30 21:46:34.444747 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:46:34.444732    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644k758g"
Oct 30 21:46:34.448218 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:46:34.448201    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651czllrv"
Oct 30 21:46:34.500740 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:46:34.500711    4473 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Oct 30 21:46:34.500740 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:46:34.500734    4473 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Oct 30 21:46:34.500996 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:46:34.500881    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"c4959aeb960c8e20c88ac2de924a35b8221494-145dcd5a-dockercfg-s24cn\""
Oct 30 21:46:34.552483 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:46:34.552448    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644k758g"]
Oct 30 21:46:34.553100 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:46:34.553079    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651czllrv"]
Oct 30 21:46:34.598363 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:46:34.598333    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"c7537007944577f64df34960ace3c69136c416-139ec370-dockercfg-2zlmm\""
Oct 30 21:46:34.611682 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:46:34.611655    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2599553b-9388-4131-8a18-115cb92730af-util\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651czllrv\" (UID: \"2599553b-9388-4131-8a18-115cb92730af\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651czllrv"
Oct 30 21:46:34.611835 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:46:34.611689    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0d74922c-6969-4422-b4c7-a692d8c1e52c-bundle\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644k758g\" (UID: \"0d74922c-6969-4422-b4c7-a692d8c1e52c\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644k758g"
Oct 30 21:46:34.611835 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:46:34.611718    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rd77l\" (UniqueName: \"kubernetes.io/projected/0d74922c-6969-4422-b4c7-a692d8c1e52c-kube-api-access-rd77l\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644k758g\" (UID: \"0d74922c-6969-4422-b4c7-a692d8c1e52c\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644k758g"
Oct 30 21:46:34.611835 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:46:34.611823    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0d74922c-6969-4422-b4c7-a692d8c1e52c-util\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644k758g\" (UID: \"0d74922c-6969-4422-b4c7-a692d8c1e52c\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644k758g"
Oct 30 21:46:34.611956 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:46:34.611843    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2599553b-9388-4131-8a18-115cb92730af-bundle\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651czllrv\" (UID: \"2599553b-9388-4131-8a18-115cb92730af\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651czllrv"
Oct 30 21:46:34.611956 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:46:34.611868    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hb6zv\" (UniqueName: \"kubernetes.io/projected/2599553b-9388-4131-8a18-115cb92730af-kube-api-access-hb6zv\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651czllrv\" (UID: \"2599553b-9388-4131-8a18-115cb92730af\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651czllrv"
Oct 30 21:46:34.712950 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:46:34.712840    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2599553b-9388-4131-8a18-115cb92730af-util\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651czllrv\" (UID: \"2599553b-9388-4131-8a18-115cb92730af\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651czllrv"
Oct 30 21:46:34.712950 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:46:34.712884    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0d74922c-6969-4422-b4c7-a692d8c1e52c-bundle\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644k758g\" (UID: \"0d74922c-6969-4422-b4c7-a692d8c1e52c\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644k758g"
Oct 30 21:46:34.712950 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:46:34.712912    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rd77l\" (UniqueName: \"kubernetes.io/projected/0d74922c-6969-4422-b4c7-a692d8c1e52c-kube-api-access-rd77l\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644k758g\" (UID: \"0d74922c-6969-4422-b4c7-a692d8c1e52c\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644k758g"
Oct 30 21:46:34.713237 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:46:34.712968    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0d74922c-6969-4422-b4c7-a692d8c1e52c-util\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644k758g\" (UID: \"0d74922c-6969-4422-b4c7-a692d8c1e52c\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644k758g"
Oct 30 21:46:34.713237 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:46:34.712989    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2599553b-9388-4131-8a18-115cb92730af-bundle\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651czllrv\" (UID: \"2599553b-9388-4131-8a18-115cb92730af\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651czllrv"
Oct 30 21:46:34.713237 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:46:34.713016    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hb6zv\" (UniqueName: \"kubernetes.io/projected/2599553b-9388-4131-8a18-115cb92730af-kube-api-access-hb6zv\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651czllrv\" (UID: \"2599553b-9388-4131-8a18-115cb92730af\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651czllrv"
Oct 30 21:46:34.713397 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:46:34.713322    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2599553b-9388-4131-8a18-115cb92730af-util\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651czllrv\" (UID: \"2599553b-9388-4131-8a18-115cb92730af\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651czllrv"
Oct 30 21:46:34.713397 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:46:34.713342    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0d74922c-6969-4422-b4c7-a692d8c1e52c-bundle\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644k758g\" (UID: \"0d74922c-6969-4422-b4c7-a692d8c1e52c\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644k758g"
Oct 30 21:46:34.713397 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:46:34.713370    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2599553b-9388-4131-8a18-115cb92730af-bundle\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651czllrv\" (UID: \"2599553b-9388-4131-8a18-115cb92730af\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651czllrv"
Oct 30 21:46:34.713493 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:46:34.713376    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0d74922c-6969-4422-b4c7-a692d8c1e52c-util\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644k758g\" (UID: \"0d74922c-6969-4422-b4c7-a692d8c1e52c\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644k758g"
Oct 30 21:46:34.771535 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:46:34.771502    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hb6zv\" (UniqueName: \"kubernetes.io/projected/2599553b-9388-4131-8a18-115cb92730af-kube-api-access-hb6zv\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651czllrv\" (UID: \"2599553b-9388-4131-8a18-115cb92730af\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651czllrv"
Oct 30 21:46:34.771680 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:46:34.771502    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rd77l\" (UniqueName: \"kubernetes.io/projected/0d74922c-6969-4422-b4c7-a692d8c1e52c-kube-api-access-rd77l\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644k758g\" (UID: \"0d74922c-6969-4422-b4c7-a692d8c1e52c\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644k758g"
Oct 30 21:46:34.840169 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:46:34.840137    4473 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-pg6b6"]
Oct 30 21:46:34.843486 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:46:34.843464    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-pg6b6"
Oct 30 21:46:34.896944 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:46:34.896878    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Oct 30 21:46:34.897129 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:46:34.896964    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Oct 30 21:46:34.897129 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:46:34.896982    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Oct 30 21:46:34.897129 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:46:34.897030    4473 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Oct 30 21:46:34.897129 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:46:34.897035    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-5jfgf\""
Oct 30 21:46:34.897350 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:46:34.897150    4473 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Oct 30 21:46:34.897350 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:46:34.897178    4473 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Oct 30 21:46:34.906208 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:46:34.906179    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-pg6b6"]
Oct 30 21:46:35.015684 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:46:35.015602    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/36fd53ff-3e4a-4581-9c87-cd1ccf42e02e-local-certs\") pod \"istiod-openshift-gateway-7876bfcb74-pg6b6\" (UID: \"36fd53ff-3e4a-4581-9c87-cd1ccf42e02e\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-pg6b6"
Oct 30 21:46:35.015684 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:46:35.015652    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/36fd53ff-3e4a-4581-9c87-cd1ccf42e02e-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7876bfcb74-pg6b6\" (UID: \"36fd53ff-3e4a-4581-9c87-cd1ccf42e02e\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-pg6b6"
Oct 30 21:46:35.015870 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:46:35.015735    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/36fd53ff-3e4a-4581-9c87-cd1ccf42e02e-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7876bfcb74-pg6b6\" (UID: \"36fd53ff-3e4a-4581-9c87-cd1ccf42e02e\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-pg6b6"
Oct 30 21:46:35.015870 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:46:35.015770    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/36fd53ff-3e4a-4581-9c87-cd1ccf42e02e-istio-kubeconfig\") pod \"istiod-openshift-gateway-7876bfcb74-pg6b6\" (UID: \"36fd53ff-3e4a-4581-9c87-cd1ccf42e02e\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-pg6b6"
Oct 30 21:46:35.015978 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:46:35.015852    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/36fd53ff-3e4a-4581-9c87-cd1ccf42e02e-cacerts\") pod \"istiod-openshift-gateway-7876bfcb74-pg6b6\" (UID: \"36fd53ff-3e4a-4581-9c87-cd1ccf42e02e\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-pg6b6"
Oct 30 21:46:35.015978 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:46:35.015896    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/36fd53ff-3e4a-4581-9c87-cd1ccf42e02e-istio-token\") pod \"istiod-openshift-gateway-7876bfcb74-pg6b6\" (UID: \"36fd53ff-3e4a-4581-9c87-cd1ccf42e02e\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-pg6b6"
Oct 30 21:46:35.015978 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:46:35.015954    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fpxq4\" (UniqueName: \"kubernetes.io/projected/36fd53ff-3e4a-4581-9c87-cd1ccf42e02e-kube-api-access-fpxq4\") pod \"istiod-openshift-gateway-7876bfcb74-pg6b6\" (UID: \"36fd53ff-3e4a-4581-9c87-cd1ccf42e02e\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-pg6b6"
Oct 30 21:46:35.061062 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:46:35.061027    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644k758g"
Oct 30 21:46:35.068107 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:46:35.068087    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651czllrv"
Oct 30 21:46:35.117346 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:46:35.117313    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/36fd53ff-3e4a-4581-9c87-cd1ccf42e02e-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7876bfcb74-pg6b6\" (UID: \"36fd53ff-3e4a-4581-9c87-cd1ccf42e02e\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-pg6b6"
Oct 30 21:46:35.117530 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:46:35.116467    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/36fd53ff-3e4a-4581-9c87-cd1ccf42e02e-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7876bfcb74-pg6b6\" (UID: \"36fd53ff-3e4a-4581-9c87-cd1ccf42e02e\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-pg6b6"
Oct 30 21:46:35.117530 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:46:35.117446    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/36fd53ff-3e4a-4581-9c87-cd1ccf42e02e-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7876bfcb74-pg6b6\" (UID: \"36fd53ff-3e4a-4581-9c87-cd1ccf42e02e\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-pg6b6"
Oct 30 21:46:35.117667 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:46:35.117523    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/36fd53ff-3e4a-4581-9c87-cd1ccf42e02e-istio-kubeconfig\") pod \"istiod-openshift-gateway-7876bfcb74-pg6b6\" (UID: \"36fd53ff-3e4a-4581-9c87-cd1ccf42e02e\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-pg6b6"
Oct 30 21:46:35.117667 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:46:35.117607    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/36fd53ff-3e4a-4581-9c87-cd1ccf42e02e-cacerts\") pod \"istiod-openshift-gateway-7876bfcb74-pg6b6\" (UID: \"36fd53ff-3e4a-4581-9c87-cd1ccf42e02e\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-pg6b6"
Oct 30 21:46:35.117780 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:46:35.117666    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/36fd53ff-3e4a-4581-9c87-cd1ccf42e02e-istio-token\") pod \"istiod-openshift-gateway-7876bfcb74-pg6b6\" (UID: \"36fd53ff-3e4a-4581-9c87-cd1ccf42e02e\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-pg6b6"
Oct 30 21:46:35.117780 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:46:35.117709    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fpxq4\" (UniqueName: \"kubernetes.io/projected/36fd53ff-3e4a-4581-9c87-cd1ccf42e02e-kube-api-access-fpxq4\") pod \"istiod-openshift-gateway-7876bfcb74-pg6b6\" (UID: \"36fd53ff-3e4a-4581-9c87-cd1ccf42e02e\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-pg6b6"
Oct 30 21:46:35.117780 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:46:35.117735    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/36fd53ff-3e4a-4581-9c87-cd1ccf42e02e-local-certs\") pod \"istiod-openshift-gateway-7876bfcb74-pg6b6\" (UID: \"36fd53ff-3e4a-4581-9c87-cd1ccf42e02e\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-pg6b6"
Oct 30 21:46:35.120752 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:46:35.120725    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/36fd53ff-3e4a-4581-9c87-cd1ccf42e02e-local-certs\") pod \"istiod-openshift-gateway-7876bfcb74-pg6b6\" (UID: \"36fd53ff-3e4a-4581-9c87-cd1ccf42e02e\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-pg6b6"
Oct 30 21:46:35.122009 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:46:35.121774    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/36fd53ff-3e4a-4581-9c87-cd1ccf42e02e-cacerts\") pod \"istiod-openshift-gateway-7876bfcb74-pg6b6\" (UID: \"36fd53ff-3e4a-4581-9c87-cd1ccf42e02e\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-pg6b6"
Oct 30 21:46:35.124150 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:46:35.123780    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/36fd53ff-3e4a-4581-9c87-cd1ccf42e02e-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7876bfcb74-pg6b6\" (UID: \"36fd53ff-3e4a-4581-9c87-cd1ccf42e02e\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-pg6b6"
Oct 30 21:46:35.124150 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:46:35.123796    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/36fd53ff-3e4a-4581-9c87-cd1ccf42e02e-istio-kubeconfig\") pod \"istiod-openshift-gateway-7876bfcb74-pg6b6\" (UID: \"36fd53ff-3e4a-4581-9c87-cd1ccf42e02e\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-pg6b6"
Oct 30 21:46:35.176374 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:46:35.176334    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/36fd53ff-3e4a-4581-9c87-cd1ccf42e02e-istio-token\") pod \"istiod-openshift-gateway-7876bfcb74-pg6b6\" (UID: \"36fd53ff-3e4a-4581-9c87-cd1ccf42e02e\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-pg6b6"
Oct 30 21:46:35.176516 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:46:35.176476    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fpxq4\" (UniqueName: \"kubernetes.io/projected/36fd53ff-3e4a-4581-9c87-cd1ccf42e02e-kube-api-access-fpxq4\") pod \"istiod-openshift-gateway-7876bfcb74-pg6b6\" (UID: \"36fd53ff-3e4a-4581-9c87-cd1ccf42e02e\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-pg6b6"
Oct 30 21:46:35.442507 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:46:35.442486    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644k758g"]
Oct 30 21:46:35.445267 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:46:35.445238    4473 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d74922c_6969_4422_b4c7_a692d8c1e52c.slice/crio-48cddc4b2a71715c5bc239a7271c80e94e8f96a2ce6a880e7a760e98907cc774 WatchSource:0}: Error finding container 48cddc4b2a71715c5bc239a7271c80e94e8f96a2ce6a880e7a760e98907cc774: Status 404 returned error can't find the container with id 48cddc4b2a71715c5bc239a7271c80e94e8f96a2ce6a880e7a760e98907cc774
Oct 30 21:46:35.454995 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:46:35.454972    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-pg6b6"
Oct 30 21:46:35.485136 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:46:35.485102    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651czllrv"]
Oct 30 21:46:35.487973 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:46:35.487909    4473 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2599553b_9388_4131_8a18_115cb92730af.slice/crio-b7df19a6b725dc6e8368996e2d64c8a394732224a45082be46ea581d05e48ac9 WatchSource:0}: Error finding container b7df19a6b725dc6e8368996e2d64c8a394732224a45082be46ea581d05e48ac9: Status 404 returned error can't find the container with id b7df19a6b725dc6e8368996e2d64c8a394732224a45082be46ea581d05e48ac9
Oct 30 21:46:35.833247 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:46:35.833224    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-pg6b6"]
Oct 30 21:46:35.835552 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:46:35.835523    4473 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36fd53ff_3e4a_4581_9c87_cd1ccf42e02e.slice/crio-b3395d2761513345d9f8af41ea8634da7fc88db86b0ffed2df792d41d42d4bab WatchSource:0}: Error finding container b3395d2761513345d9f8af41ea8634da7fc88db86b0ffed2df792d41d42d4bab: Status 404 returned error can't find the container with id b3395d2761513345d9f8af41ea8634da7fc88db86b0ffed2df792d41d42d4bab
Oct 30 21:46:36.332731 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:46:36.332695    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-pg6b6" event={"ID":"36fd53ff-3e4a-4581-9c87-cd1ccf42e02e","Type":"ContainerStarted","Data":"b3395d2761513345d9f8af41ea8634da7fc88db86b0ffed2df792d41d42d4bab"}
Oct 30 21:46:36.333894 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:46:36.333868    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644k758g" event={"ID":"0d74922c-6969-4422-b4c7-a692d8c1e52c","Type":"ContainerStarted","Data":"48cddc4b2a71715c5bc239a7271c80e94e8f96a2ce6a880e7a760e98907cc774"}
Oct 30 21:46:36.335020 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:46:36.334999    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651czllrv" event={"ID":"2599553b-9388-4131-8a18-115cb92730af","Type":"ContainerStarted","Data":"b7df19a6b725dc6e8368996e2d64c8a394732224a45082be46ea581d05e48ac9"}
Oct 30 21:46:40.099053 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:46:40.099010    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:46:40.099441 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:46:40.099095    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:46:40.359489 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:46:40.359399    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-pg6b6" event={"ID":"36fd53ff-3e4a-4581-9c87-cd1ccf42e02e","Type":"ContainerStarted","Data":"e8670e41a0c5435b30e6a6496d19ab018f2c03793bd03025842f7a945d8af40a"}
Oct 30 21:46:40.360195 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:46:40.360158    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-pg6b6"
Oct 30 21:46:40.580155 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:46:40.580102    4473 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-pg6b6" podStartSLOduration=2.319606921 podStartE2EDuration="6.580085352s" totalImagesPullingTime="4.260478431s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:46:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:46:40.528993026 +0000 UTC m=+246.302361514" watchObservedRunningTime="2026-10-30 21:46:40.580085352 +0000 UTC m=+246.353453839"
Oct 30 21:46:41.365911 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:46:41.365875    4473 patch_prober.go:29] interesting pod/istiod-openshift-gateway-7876bfcb74-pg6b6 container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 21:46:41.366419 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:46:41.365965    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-pg6b6" podUID="36fd53ff-3e4a-4581-9c87-cd1ccf42e02e" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 21:46:42.369057 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:46:42.369024    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-pg6b6"
Oct 30 21:46:43.373113 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:46:43.373082    4473 generic.go:356] "Generic (PLEG): container finished" podID="0d74922c-6969-4422-b4c7-a692d8c1e52c" containerID="bf4c536d6fe7a35bb6420398e1ce84a13674bde8fc90a1441852b1d6eebaa4e8" exitCode=0
Oct 30 21:46:43.373555 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:46:43.373176    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644k758g" event={"ID":"0d74922c-6969-4422-b4c7-a692d8c1e52c","Type":"ContainerDied","Data":"bf4c536d6fe7a35bb6420398e1ce84a13674bde8fc90a1441852b1d6eebaa4e8"}
Oct 30 21:46:43.374693 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:46:43.374671    4473 generic.go:356] "Generic (PLEG): container finished" podID="2599553b-9388-4131-8a18-115cb92730af" containerID="f73a7270aa60923de95c628411562c24958846a0a2e636f2046f9fcb88afd4b4" exitCode=0
Oct 30 21:46:43.374827 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:46:43.374762    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651czllrv" event={"ID":"2599553b-9388-4131-8a18-115cb92730af","Type":"ContainerDied","Data":"f73a7270aa60923de95c628411562c24958846a0a2e636f2046f9fcb88afd4b4"}
Oct 30 21:46:47.392077 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:46:47.392046    4473 generic.go:356] "Generic (PLEG): container finished" podID="0d74922c-6969-4422-b4c7-a692d8c1e52c" containerID="cd91cd611bdd40a190ad881d2b50de24b6012f724b745406ee1dac25cfad6c6e" exitCode=0
Oct 30 21:46:47.392533 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:46:47.392138    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644k758g" event={"ID":"0d74922c-6969-4422-b4c7-a692d8c1e52c","Type":"ContainerDied","Data":"cd91cd611bdd40a190ad881d2b50de24b6012f724b745406ee1dac25cfad6c6e"}
Oct 30 21:46:47.393957 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:46:47.393843    4473 generic.go:356] "Generic (PLEG): container finished" podID="2599553b-9388-4131-8a18-115cb92730af" containerID="3f59c7916d4b7fe1b27711edb3c1c80b3128268d02529920dc443d728f48ab84" exitCode=0
Oct 30 21:46:47.393957 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:46:47.393904    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651czllrv" event={"ID":"2599553b-9388-4131-8a18-115cb92730af","Type":"ContainerDied","Data":"3f59c7916d4b7fe1b27711edb3c1c80b3128268d02529920dc443d728f48ab84"}
Oct 30 21:46:50.332322 ip-10-0-140-18 kubenswrapper[4473]: E1030 21:46:50.332283    4473 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fncvg" containerName="extractor"
Oct 30 21:46:55.432484 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:46:55.432448    4473 generic.go:356] "Generic (PLEG): container finished" podID="0d74922c-6969-4422-b4c7-a692d8c1e52c" containerID="c8ad7199f1393688db6005ac3848c19b8501e41fe6b1852a0fc23e0ab3544ff6" exitCode=0
Oct 30 21:46:55.432970 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:46:55.432526    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644k758g" event={"ID":"0d74922c-6969-4422-b4c7-a692d8c1e52c","Type":"ContainerDied","Data":"c8ad7199f1393688db6005ac3848c19b8501e41fe6b1852a0fc23e0ab3544ff6"}
Oct 30 21:46:56.438676 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:46:56.438642    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651czllrv" event={"ID":"2599553b-9388-4131-8a18-115cb92730af","Type":"ContainerStarted","Data":"787689b4e3d9cbdcc4db120f0e70033f5307437618e9d3ec52c5ebbb8aada280"}
Oct 30 21:46:56.560303 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:46:56.560258    4473 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651czllrv" podStartSLOduration=3.337129555 podStartE2EDuration="22.560243424s" totalImagesPullingTime="19.223113869s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:46:34 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:46:56.55917107 +0000 UTC m=+262.332539557" watchObservedRunningTime="2026-10-30 21:46:56.560243424 +0000 UTC m=+262.333611888"
Oct 30 21:46:56.594352 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:46:56.594323    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644k758g"
Oct 30 21:46:56.703641 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:46:56.703610    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0d74922c-6969-4422-b4c7-a692d8c1e52c-util\" (UniqueName: \"kubernetes.io/empty-dir/0d74922c-6969-4422-b4c7-a692d8c1e52c-util\") pod \"0d74922c-6969-4422-b4c7-a692d8c1e52c\" (UID: \"0d74922c-6969-4422-b4c7-a692d8c1e52c\") "
Oct 30 21:46:56.703806 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:46:56.703704    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0d74922c-6969-4422-b4c7-a692d8c1e52c-kube-api-access-rd77l\" (UniqueName: \"kubernetes.io/projected/0d74922c-6969-4422-b4c7-a692d8c1e52c-kube-api-access-rd77l\") pod \"0d74922c-6969-4422-b4c7-a692d8c1e52c\" (UID: \"0d74922c-6969-4422-b4c7-a692d8c1e52c\") "
Oct 30 21:46:56.703806 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:46:56.703727    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0d74922c-6969-4422-b4c7-a692d8c1e52c-bundle\" (UniqueName: \"kubernetes.io/empty-dir/0d74922c-6969-4422-b4c7-a692d8c1e52c-bundle\") pod \"0d74922c-6969-4422-b4c7-a692d8c1e52c\" (UID: \"0d74922c-6969-4422-b4c7-a692d8c1e52c\") "
Oct 30 21:46:56.704240 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:46:56.704205    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d74922c-6969-4422-b4c7-a692d8c1e52c-bundle" pod "0d74922c-6969-4422-b4c7-a692d8c1e52c" (UID: "0d74922c-6969-4422-b4c7-a692d8c1e52c"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:46:56.706259 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:46:56.706230    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0d74922c-6969-4422-b4c7-a692d8c1e52c-kube-api-access-rd77l" pod "0d74922c-6969-4422-b4c7-a692d8c1e52c" (UID: "0d74922c-6969-4422-b4c7-a692d8c1e52c"). InnerVolumeSpecName "kube-api-access-rd77l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:46:56.708753 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:46:56.708729    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d74922c-6969-4422-b4c7-a692d8c1e52c-util" pod "0d74922c-6969-4422-b4c7-a692d8c1e52c" (UID: "0d74922c-6969-4422-b4c7-a692d8c1e52c"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:46:56.804974 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:46:56.804875    4473 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rd77l\" (UniqueName: \"kubernetes.io/projected/0d74922c-6969-4422-b4c7-a692d8c1e52c-kube-api-access-rd77l\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:46:56.804974 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:46:56.804907    4473 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0d74922c-6969-4422-b4c7-a692d8c1e52c-bundle\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:46:56.804974 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:46:56.804936    4473 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0d74922c-6969-4422-b4c7-a692d8c1e52c-util\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:46:57.445179 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:46:57.445146    4473 generic.go:356] "Generic (PLEG): container finished" podID="2599553b-9388-4131-8a18-115cb92730af" containerID="787689b4e3d9cbdcc4db120f0e70033f5307437618e9d3ec52c5ebbb8aada280" exitCode=0
Oct 30 21:46:57.445606 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:46:57.445240    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651czllrv" event={"ID":"2599553b-9388-4131-8a18-115cb92730af","Type":"ContainerDied","Data":"787689b4e3d9cbdcc4db120f0e70033f5307437618e9d3ec52c5ebbb8aada280"}
Oct 30 21:46:57.446970 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:46:57.446946    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644k758g" event={"ID":"0d74922c-6969-4422-b4c7-a692d8c1e52c","Type":"ContainerDied","Data":"48cddc4b2a71715c5bc239a7271c80e94e8f96a2ce6a880e7a760e98907cc774"}
Oct 30 21:46:57.447103 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:46:57.446982    4473 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="48cddc4b2a71715c5bc239a7271c80e94e8f96a2ce6a880e7a760e98907cc774"
Oct 30 21:46:57.447103 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:46:57.446990    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644k758g"
Oct 30 21:46:58.600586 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:46:58.600549    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651czllrv"
Oct 30 21:46:58.721562 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:46:58.721475    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2599553b-9388-4131-8a18-115cb92730af-kube-api-access-hb6zv\" (UniqueName: \"kubernetes.io/projected/2599553b-9388-4131-8a18-115cb92730af-kube-api-access-hb6zv\") pod \"2599553b-9388-4131-8a18-115cb92730af\" (UID: \"2599553b-9388-4131-8a18-115cb92730af\") "
Oct 30 21:46:58.721562 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:46:58.721531    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2599553b-9388-4131-8a18-115cb92730af-bundle\" (UniqueName: \"kubernetes.io/empty-dir/2599553b-9388-4131-8a18-115cb92730af-bundle\") pod \"2599553b-9388-4131-8a18-115cb92730af\" (UID: \"2599553b-9388-4131-8a18-115cb92730af\") "
Oct 30 21:46:58.721754 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:46:58.721622    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2599553b-9388-4131-8a18-115cb92730af-util\" (UniqueName: \"kubernetes.io/empty-dir/2599553b-9388-4131-8a18-115cb92730af-util\") pod \"2599553b-9388-4131-8a18-115cb92730af\" (UID: \"2599553b-9388-4131-8a18-115cb92730af\") "
Oct 30 21:46:58.722280 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:46:58.722200    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2599553b-9388-4131-8a18-115cb92730af-bundle" pod "2599553b-9388-4131-8a18-115cb92730af" (UID: "2599553b-9388-4131-8a18-115cb92730af"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:46:58.723829 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:46:58.723801    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2599553b-9388-4131-8a18-115cb92730af-kube-api-access-hb6zv" pod "2599553b-9388-4131-8a18-115cb92730af" (UID: "2599553b-9388-4131-8a18-115cb92730af"). InnerVolumeSpecName "kube-api-access-hb6zv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:46:58.726273 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:46:58.726246    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2599553b-9388-4131-8a18-115cb92730af-util" pod "2599553b-9388-4131-8a18-115cb92730af" (UID: "2599553b-9388-4131-8a18-115cb92730af"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:46:58.823003 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:46:58.822965    4473 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2599553b-9388-4131-8a18-115cb92730af-util\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:46:58.823003 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:46:58.823003    4473 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hb6zv\" (UniqueName: \"kubernetes.io/projected/2599553b-9388-4131-8a18-115cb92730af-kube-api-access-hb6zv\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:46:58.823223 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:46:58.823016    4473 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2599553b-9388-4131-8a18-115cb92730af-bundle\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:46:59.456284 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:46:59.456251    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651czllrv"
Oct 30 21:46:59.456284 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:46:59.456273    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651czllrv" event={"ID":"2599553b-9388-4131-8a18-115cb92730af","Type":"ContainerDied","Data":"b7df19a6b725dc6e8368996e2d64c8a394732224a45082be46ea581d05e48ac9"}
Oct 30 21:46:59.456487 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:46:59.456317    4473 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b7df19a6b725dc6e8368996e2d64c8a394732224a45082be46ea581d05e48ac9"
Oct 30 21:47:20.322162 ip-10-0-140-18 kubenswrapper[4473]: E1030 21:47:20.322119    4473 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fncvg" containerName="extractor"
Oct 30 21:47:39.401523 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:47:39.401482    4473 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-c8dcc6d7b-pfjdk"]
Oct 30 21:47:39.402049 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:47:39.401788    4473 cpu_manager.go:414] "containerMap: removing container" podUID="0d74922c-6969-4422-b4c7-a692d8c1e52c" containerName="extract"
Oct 30 21:47:39.402049 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:47:39.401799    4473 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0d74922c-6969-4422-b4c7-a692d8c1e52c" containerName="extract"
Oct 30 21:47:39.402049 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:47:39.401828    4473 cpu_manager.go:414] "containerMap: removing container" podUID="0d74922c-6969-4422-b4c7-a692d8c1e52c" containerName="util"
Oct 30 21:47:39.402049 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:47:39.401833    4473 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0d74922c-6969-4422-b4c7-a692d8c1e52c" containerName="util"
Oct 30 21:47:39.402049 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:47:39.401843    4473 cpu_manager.go:414] "containerMap: removing container" podUID="2599553b-9388-4131-8a18-115cb92730af" containerName="extract"
Oct 30 21:47:39.402049 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:47:39.401848    4473 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2599553b-9388-4131-8a18-115cb92730af" containerName="extract"
Oct 30 21:47:39.402049 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:47:39.401856    4473 cpu_manager.go:414] "containerMap: removing container" podUID="2599553b-9388-4131-8a18-115cb92730af" containerName="util"
Oct 30 21:47:39.402049 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:47:39.401861    4473 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2599553b-9388-4131-8a18-115cb92730af" containerName="util"
Oct 30 21:47:39.402049 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:47:39.401867    4473 cpu_manager.go:414] "containerMap: removing container" podUID="0d74922c-6969-4422-b4c7-a692d8c1e52c" containerName="pull"
Oct 30 21:47:39.402049 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:47:39.401870    4473 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0d74922c-6969-4422-b4c7-a692d8c1e52c" containerName="pull"
Oct 30 21:47:39.402049 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:47:39.401878    4473 cpu_manager.go:414] "containerMap: removing container" podUID="2599553b-9388-4131-8a18-115cb92730af" containerName="pull"
Oct 30 21:47:39.402049 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:47:39.401882    4473 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2599553b-9388-4131-8a18-115cb92730af" containerName="pull"
Oct 30 21:47:39.402049 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:47:39.401956    4473 memory_manager.go:365] "RemoveStaleState removing state" podUID="2599553b-9388-4131-8a18-115cb92730af" containerName="extract"
Oct 30 21:47:39.402049 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:47:39.401967    4473 memory_manager.go:365] "RemoveStaleState removing state" podUID="0d74922c-6969-4422-b4c7-a692d8c1e52c" containerName="extract"
Oct 30 21:47:39.403975 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:47:39.403958    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-c8dcc6d7b-pfjdk"
Oct 30 21:47:39.452729 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:47:39.452680    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e059ef20-34be-4dcb-8a0f-1ed385ceaad3-webhook-cert\") pod \"opendatahub-operator-controller-manager-c8dcc6d7b-pfjdk\" (UID: \"e059ef20-34be-4dcb-8a0f-1ed385ceaad3\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-c8dcc6d7b-pfjdk"
Oct 30 21:47:39.452965 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:47:39.452870    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e059ef20-34be-4dcb-8a0f-1ed385ceaad3-manifests\") pod \"opendatahub-operator-controller-manager-c8dcc6d7b-pfjdk\" (UID: \"e059ef20-34be-4dcb-8a0f-1ed385ceaad3\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-c8dcc6d7b-pfjdk"
Oct 30 21:47:39.452965 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:47:39.452910    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/e059ef20-34be-4dcb-8a0f-1ed385ceaad3-metrics-cert\") pod \"opendatahub-operator-controller-manager-c8dcc6d7b-pfjdk\" (UID: \"e059ef20-34be-4dcb-8a0f-1ed385ceaad3\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-c8dcc6d7b-pfjdk"
Oct 30 21:47:39.453115 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:47:39.452973    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fvmbb\" (UniqueName: \"kubernetes.io/projected/e059ef20-34be-4dcb-8a0f-1ed385ceaad3-kube-api-access-fvmbb\") pod \"opendatahub-operator-controller-manager-c8dcc6d7b-pfjdk\" (UID: \"e059ef20-34be-4dcb-8a0f-1ed385ceaad3\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-c8dcc6d7b-pfjdk"
Oct 30 21:47:39.453115 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:47:39.453080    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e059ef20-34be-4dcb-8a0f-1ed385ceaad3-tmp\") pod \"opendatahub-operator-controller-manager-c8dcc6d7b-pfjdk\" (UID: \"e059ef20-34be-4dcb-8a0f-1ed385ceaad3\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-c8dcc6d7b-pfjdk"
Oct 30 21:47:39.453115 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:47:39.453107    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/e059ef20-34be-4dcb-8a0f-1ed385ceaad3-apiservice-cert\") pod \"opendatahub-operator-controller-manager-c8dcc6d7b-pfjdk\" (UID: \"e059ef20-34be-4dcb-8a0f-1ed385ceaad3\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-c8dcc6d7b-pfjdk"
Oct 30 21:47:39.527007 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:47:39.526971    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-c8dcc6d7b-pfjdk"]
Oct 30 21:47:39.543457 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:47:39.543429    4473 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub-operator-system\"/\"openshift-service-ca.crt\""
Oct 30 21:47:39.543716 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:47:39.543698    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-webhook-cert\""
Oct 30 21:47:39.544255 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:47:39.544236    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-manager-service-cert\""
Oct 30 21:47:39.544642 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:47:39.544616    4473 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub-operator-system\"/\"kube-root-ca.crt\""
Oct 30 21:47:39.544783 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:47:39.544693    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-metrics-tls\""
Oct 30 21:47:39.544856 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:47:39.544812    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-manager-dockercfg-w7mjw\""
Oct 30 21:47:39.554115 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:47:39.554085    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e059ef20-34be-4dcb-8a0f-1ed385ceaad3-webhook-cert\") pod \"opendatahub-operator-controller-manager-c8dcc6d7b-pfjdk\" (UID: \"e059ef20-34be-4dcb-8a0f-1ed385ceaad3\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-c8dcc6d7b-pfjdk"
Oct 30 21:47:39.554245 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:47:39.554128    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e059ef20-34be-4dcb-8a0f-1ed385ceaad3-manifests\") pod \"opendatahub-operator-controller-manager-c8dcc6d7b-pfjdk\" (UID: \"e059ef20-34be-4dcb-8a0f-1ed385ceaad3\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-c8dcc6d7b-pfjdk"
Oct 30 21:47:39.554245 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:47:39.554155    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/e059ef20-34be-4dcb-8a0f-1ed385ceaad3-metrics-cert\") pod \"opendatahub-operator-controller-manager-c8dcc6d7b-pfjdk\" (UID: \"e059ef20-34be-4dcb-8a0f-1ed385ceaad3\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-c8dcc6d7b-pfjdk"
Oct 30 21:47:39.554245 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:47:39.554179    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fvmbb\" (UniqueName: \"kubernetes.io/projected/e059ef20-34be-4dcb-8a0f-1ed385ceaad3-kube-api-access-fvmbb\") pod \"opendatahub-operator-controller-manager-c8dcc6d7b-pfjdk\" (UID: \"e059ef20-34be-4dcb-8a0f-1ed385ceaad3\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-c8dcc6d7b-pfjdk"
Oct 30 21:47:39.554245 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:47:39.554201    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e059ef20-34be-4dcb-8a0f-1ed385ceaad3-tmp\") pod \"opendatahub-operator-controller-manager-c8dcc6d7b-pfjdk\" (UID: \"e059ef20-34be-4dcb-8a0f-1ed385ceaad3\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-c8dcc6d7b-pfjdk"
Oct 30 21:47:39.554245 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:47:39.554226    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/e059ef20-34be-4dcb-8a0f-1ed385ceaad3-apiservice-cert\") pod \"opendatahub-operator-controller-manager-c8dcc6d7b-pfjdk\" (UID: \"e059ef20-34be-4dcb-8a0f-1ed385ceaad3\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-c8dcc6d7b-pfjdk"
Oct 30 21:47:39.554595 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:47:39.554576    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e059ef20-34be-4dcb-8a0f-1ed385ceaad3-manifests\") pod \"opendatahub-operator-controller-manager-c8dcc6d7b-pfjdk\" (UID: \"e059ef20-34be-4dcb-8a0f-1ed385ceaad3\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-c8dcc6d7b-pfjdk"
Oct 30 21:47:39.556828 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:47:39.556799    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e059ef20-34be-4dcb-8a0f-1ed385ceaad3-tmp\") pod \"opendatahub-operator-controller-manager-c8dcc6d7b-pfjdk\" (UID: \"e059ef20-34be-4dcb-8a0f-1ed385ceaad3\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-c8dcc6d7b-pfjdk"
Oct 30 21:47:39.557082 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:47:39.556858    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e059ef20-34be-4dcb-8a0f-1ed385ceaad3-webhook-cert\") pod \"opendatahub-operator-controller-manager-c8dcc6d7b-pfjdk\" (UID: \"e059ef20-34be-4dcb-8a0f-1ed385ceaad3\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-c8dcc6d7b-pfjdk"
Oct 30 21:47:39.557082 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:47:39.556893    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/e059ef20-34be-4dcb-8a0f-1ed385ceaad3-apiservice-cert\") pod \"opendatahub-operator-controller-manager-c8dcc6d7b-pfjdk\" (UID: \"e059ef20-34be-4dcb-8a0f-1ed385ceaad3\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-c8dcc6d7b-pfjdk"
Oct 30 21:47:39.557082 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:47:39.556948    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/e059ef20-34be-4dcb-8a0f-1ed385ceaad3-metrics-cert\") pod \"opendatahub-operator-controller-manager-c8dcc6d7b-pfjdk\" (UID: \"e059ef20-34be-4dcb-8a0f-1ed385ceaad3\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-c8dcc6d7b-pfjdk"
Oct 30 21:47:39.612778 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:47:39.612744    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fvmbb\" (UniqueName: \"kubernetes.io/projected/e059ef20-34be-4dcb-8a0f-1ed385ceaad3-kube-api-access-fvmbb\") pod \"opendatahub-operator-controller-manager-c8dcc6d7b-pfjdk\" (UID: \"e059ef20-34be-4dcb-8a0f-1ed385ceaad3\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-c8dcc6d7b-pfjdk"
Oct 30 21:47:39.717194 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:47:39.717090    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-c8dcc6d7b-pfjdk"
Oct 30 21:47:40.203071 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:47:40.203042    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-c8dcc6d7b-pfjdk"]
Oct 30 21:47:40.204167 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:47:40.204144    4473 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode059ef20_34be_4dcb_8a0f_1ed385ceaad3.slice/crio-e06d7466b6b5f110fee38360ce5306198bffc9a37bc4b940c9197b2e17a4c0dd WatchSource:0}: Error finding container e06d7466b6b5f110fee38360ce5306198bffc9a37bc4b940c9197b2e17a4c0dd: Status 404 returned error can't find the container with id e06d7466b6b5f110fee38360ce5306198bffc9a37bc4b940c9197b2e17a4c0dd
Oct 30 21:47:40.206139 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:47:40.206114    4473 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 21:47:40.613151 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:47:40.613048    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-c8dcc6d7b-pfjdk" event={"ID":"e059ef20-34be-4dcb-8a0f-1ed385ceaad3","Type":"ContainerStarted","Data":"e06d7466b6b5f110fee38360ce5306198bffc9a37bc4b940c9197b2e17a4c0dd"}
Oct 30 21:47:43.629545 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:47:43.629504    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-c8dcc6d7b-pfjdk" event={"ID":"e059ef20-34be-4dcb-8a0f-1ed385ceaad3","Type":"ContainerStarted","Data":"b17f4c24d57fe7cb96bcd4dfb59cb6a6299b609a8d3c50919a61f3f3f58ea472"}
Oct 30 21:47:44.635520 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:47:44.635483    4473 generic.go:356] "Generic (PLEG): container finished" podID="e059ef20-34be-4dcb-8a0f-1ed385ceaad3" containerID="b17f4c24d57fe7cb96bcd4dfb59cb6a6299b609a8d3c50919a61f3f3f58ea472" exitCode=0
Oct 30 21:47:44.635954 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:47:44.635564    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-c8dcc6d7b-pfjdk" event={"ID":"e059ef20-34be-4dcb-8a0f-1ed385ceaad3","Type":"ContainerDied","Data":"b17f4c24d57fe7cb96bcd4dfb59cb6a6299b609a8d3c50919a61f3f3f58ea472"}
Oct 30 21:47:45.641334 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:47:45.641292    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-c8dcc6d7b-pfjdk" event={"ID":"e059ef20-34be-4dcb-8a0f-1ed385ceaad3","Type":"ContainerStarted","Data":"c2d656d03928ec95746316f002d2ac5c5dbd4c4fefb9b35dde83fe81bc8f506e"}
Oct 30 21:47:45.641746 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:47:45.641405    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-c8dcc6d7b-pfjdk"
Oct 30 21:47:45.827814 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:47:45.827713    4473 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-c8dcc6d7b-pfjdk" podStartSLOduration=2.202894308 podStartE2EDuration="6.827700293s" totalImagesPullingTime="4.175125269s" totalInitContainerRuntime="449.680716ms" isStatefulPod=false podCreationTimestamp="2026-10-30 21:47:39 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:47:45.825692652 +0000 UTC m=+311.599061173" watchObservedRunningTime="2026-10-30 21:47:45.827700293 +0000 UTC m=+311.601068828"
Oct 30 21:47:50.322678 ip-10-0-140-18 kubenswrapper[4473]: E1030 21:47:50.322639    4473 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fncvg" containerName="extractor"
Oct 30 21:47:56.648004 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:47:56.647966    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-c8dcc6d7b-pfjdk"
Oct 30 21:48:16.770954 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:48:16.770903    4473 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-5cbcf87c76-fnvrz"]
Oct 30 21:48:16.777147 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:48:16.777123    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5cbcf87c76-fnvrz"
Oct 30 21:48:16.879378 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:48:16.879347    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5cbcf87c76-fnvrz"]
Oct 30 21:48:16.881616 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:48:16.881597    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f1348ad3-7562-42e5-a9fb-5aa36f6eb9c5-tls-certs\") pod \"kube-auth-proxy-5cbcf87c76-fnvrz\" (UID: \"f1348ad3-7562-42e5-a9fb-5aa36f6eb9c5\") " pod="openshift-ingress/kube-auth-proxy-5cbcf87c76-fnvrz"
Oct 30 21:48:16.881708 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:48:16.881643    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-774br\" (UniqueName: \"kubernetes.io/projected/f1348ad3-7562-42e5-a9fb-5aa36f6eb9c5-kube-api-access-774br\") pod \"kube-auth-proxy-5cbcf87c76-fnvrz\" (UID: \"f1348ad3-7562-42e5-a9fb-5aa36f6eb9c5\") " pod="openshift-ingress/kube-auth-proxy-5cbcf87c76-fnvrz"
Oct 30 21:48:16.881786 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:48:16.881762    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f1348ad3-7562-42e5-a9fb-5aa36f6eb9c5-tmp\") pod \"kube-auth-proxy-5cbcf87c76-fnvrz\" (UID: \"f1348ad3-7562-42e5-a9fb-5aa36f6eb9c5\") " pod="openshift-ingress/kube-auth-proxy-5cbcf87c76-fnvrz"
Oct 30 21:48:16.903233 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:48:16.903208    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Oct 30 21:48:16.903233 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:48:16.903233    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-x8f47\""
Oct 30 21:48:16.903429 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:48:16.903281    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Oct 30 21:48:16.982800 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:48:16.982770    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f1348ad3-7562-42e5-a9fb-5aa36f6eb9c5-tls-certs\") pod \"kube-auth-proxy-5cbcf87c76-fnvrz\" (UID: \"f1348ad3-7562-42e5-a9fb-5aa36f6eb9c5\") " pod="openshift-ingress/kube-auth-proxy-5cbcf87c76-fnvrz"
Oct 30 21:48:16.982982 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:48:16.982817    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-774br\" (UniqueName: \"kubernetes.io/projected/f1348ad3-7562-42e5-a9fb-5aa36f6eb9c5-kube-api-access-774br\") pod \"kube-auth-proxy-5cbcf87c76-fnvrz\" (UID: \"f1348ad3-7562-42e5-a9fb-5aa36f6eb9c5\") " pod="openshift-ingress/kube-auth-proxy-5cbcf87c76-fnvrz"
Oct 30 21:48:16.982982 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:48:16.982853    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f1348ad3-7562-42e5-a9fb-5aa36f6eb9c5-tmp\") pod \"kube-auth-proxy-5cbcf87c76-fnvrz\" (UID: \"f1348ad3-7562-42e5-a9fb-5aa36f6eb9c5\") " pod="openshift-ingress/kube-auth-proxy-5cbcf87c76-fnvrz"
Oct 30 21:48:16.985199 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:48:16.985135    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f1348ad3-7562-42e5-a9fb-5aa36f6eb9c5-tmp\") pod \"kube-auth-proxy-5cbcf87c76-fnvrz\" (UID: \"f1348ad3-7562-42e5-a9fb-5aa36f6eb9c5\") " pod="openshift-ingress/kube-auth-proxy-5cbcf87c76-fnvrz"
Oct 30 21:48:16.985342 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:48:16.985323    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f1348ad3-7562-42e5-a9fb-5aa36f6eb9c5-tls-certs\") pod \"kube-auth-proxy-5cbcf87c76-fnvrz\" (UID: \"f1348ad3-7562-42e5-a9fb-5aa36f6eb9c5\") " pod="openshift-ingress/kube-auth-proxy-5cbcf87c76-fnvrz"
Oct 30 21:48:17.042389 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:48:17.042318    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-774br\" (UniqueName: \"kubernetes.io/projected/f1348ad3-7562-42e5-a9fb-5aa36f6eb9c5-kube-api-access-774br\") pod \"kube-auth-proxy-5cbcf87c76-fnvrz\" (UID: \"f1348ad3-7562-42e5-a9fb-5aa36f6eb9c5\") " pod="openshift-ingress/kube-auth-proxy-5cbcf87c76-fnvrz"
Oct 30 21:48:17.087965 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:48:17.087931    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5cbcf87c76-fnvrz"
Oct 30 21:48:17.456460 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:48:17.456431    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5cbcf87c76-fnvrz"]
Oct 30 21:48:17.458590 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:48:17.458562    4473 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1348ad3_7562_42e5_a9fb_5aa36f6eb9c5.slice/crio-ce4be8a41448888a220ed61973d6f26976ab1382043578a0cf528997abaaaff3 WatchSource:0}: Error finding container ce4be8a41448888a220ed61973d6f26976ab1382043578a0cf528997abaaaff3: Status 404 returned error can't find the container with id ce4be8a41448888a220ed61973d6f26976ab1382043578a0cf528997abaaaff3
Oct 30 21:48:17.774797 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:48:17.774715    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5cbcf87c76-fnvrz" event={"ID":"f1348ad3-7562-42e5-a9fb-5aa36f6eb9c5","Type":"ContainerStarted","Data":"ce4be8a41448888a220ed61973d6f26976ab1382043578a0cf528997abaaaff3"}
Oct 30 21:48:19.563154 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:48:19.562024    4473 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-scrsv"]
Oct 30 21:48:19.567675 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:48:19.567634    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-74dfcfb894-scrsv"
Oct 30 21:48:19.622688 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:48:19.622646    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-scrsv"]
Oct 30 21:48:19.671091 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:48:19.671053    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-6d9db\""
Oct 30 21:48:19.671091 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:48:19.671058    4473 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Oct 30 21:48:19.671352 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:48:19.671096    4473 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Oct 30 21:48:19.671352 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:48:19.671104    4473 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Oct 30 21:48:19.705462 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:48:19.705434    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/4fa01c3e-1438-4945-9353-d5058935606e-config-file\") pod \"limitador-limitador-74dfcfb894-scrsv\" (UID: \"4fa01c3e-1438-4945-9353-d5058935606e\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-scrsv"
Oct 30 21:48:19.705649 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:48:19.705515    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ghx6k\" (UniqueName: \"kubernetes.io/projected/4fa01c3e-1438-4945-9353-d5058935606e-kube-api-access-ghx6k\") pod \"limitador-limitador-74dfcfb894-scrsv\" (UID: \"4fa01c3e-1438-4945-9353-d5058935606e\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-scrsv"
Oct 30 21:48:19.804099 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:48:19.804067    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-scrsv"]
Oct 30 21:48:19.806136 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:48:19.806099    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/4fa01c3e-1438-4945-9353-d5058935606e-config-file\") pod \"limitador-limitador-74dfcfb894-scrsv\" (UID: \"4fa01c3e-1438-4945-9353-d5058935606e\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-scrsv"
Oct 30 21:48:19.806283 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:48:19.806191    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ghx6k\" (UniqueName: \"kubernetes.io/projected/4fa01c3e-1438-4945-9353-d5058935606e-kube-api-access-ghx6k\") pod \"limitador-limitador-74dfcfb894-scrsv\" (UID: \"4fa01c3e-1438-4945-9353-d5058935606e\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-scrsv"
Oct 30 21:48:19.806741 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:48:19.806716    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/4fa01c3e-1438-4945-9353-d5058935606e-config-file\") pod \"limitador-limitador-74dfcfb894-scrsv\" (UID: \"4fa01c3e-1438-4945-9353-d5058935606e\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-scrsv"
Oct 30 21:48:19.865728 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:48:19.865696    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ghx6k\" (UniqueName: \"kubernetes.io/projected/4fa01c3e-1438-4945-9353-d5058935606e-kube-api-access-ghx6k\") pod \"limitador-limitador-74dfcfb894-scrsv\" (UID: \"4fa01c3e-1438-4945-9353-d5058935606e\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-scrsv"
Oct 30 21:48:19.881414 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:48:19.881386    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-74dfcfb894-scrsv"
Oct 30 21:48:20.261536 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:48:20.261515    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-scrsv"]
Oct 30 21:48:20.263639 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:48:20.263572    4473 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4fa01c3e_1438_4945_9353_d5058935606e.slice/crio-835f8c4f34d5a412aaaf2ed0695f00c8aaab51ea35fd158709728da71ba8284b WatchSource:0}: Error finding container 835f8c4f34d5a412aaaf2ed0695f00c8aaab51ea35fd158709728da71ba8284b: Status 404 returned error can't find the container with id 835f8c4f34d5a412aaaf2ed0695f00c8aaab51ea35fd158709728da71ba8284b
Oct 30 21:48:20.336230 ip-10-0-140-18 kubenswrapper[4473]: E1030 21:48:20.335687    4473 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fncvg" containerName="extractor"
Oct 30 21:48:20.794389 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:48:20.794343    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5cbcf87c76-fnvrz" event={"ID":"f1348ad3-7562-42e5-a9fb-5aa36f6eb9c5","Type":"ContainerStarted","Data":"c44f0cc195165be54edca70796c7235c4363cc6d62b5759e1b1f8babcb9588c0"}
Oct 30 21:48:20.795560 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:48:20.795533    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-74dfcfb894-scrsv" event={"ID":"4fa01c3e-1438-4945-9353-d5058935606e","Type":"ContainerStarted","Data":"835f8c4f34d5a412aaaf2ed0695f00c8aaab51ea35fd158709728da71ba8284b"}
Oct 30 21:48:20.924140 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:48:20.920344    4473 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-5cbcf87c76-fnvrz" podStartSLOduration=2.160162267 podStartE2EDuration="4.920326258s" totalImagesPullingTime="2.760163991s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:48:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:48:20.919205313 +0000 UTC m=+346.692573803" watchObservedRunningTime="2026-10-30 21:48:20.920326258 +0000 UTC m=+346.693694745"
Oct 30 21:48:24.816906 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:48:24.816868    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-74dfcfb894-scrsv" event={"ID":"4fa01c3e-1438-4945-9353-d5058935606e","Type":"ContainerStarted","Data":"32f49ebd52897363aa15fccbc59172d190e39f74ab6fa6156e2977d1c28fab76"}
Oct 30 21:48:24.817381 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:48:24.817046    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-74dfcfb894-scrsv"
Oct 30 21:48:24.934894 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:48:24.934834    4473 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-74dfcfb894-scrsv" podStartSLOduration=2.30113404 podStartE2EDuration="5.934817946s" totalImagesPullingTime="3.633683906s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:48:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:48:24.933696045 +0000 UTC m=+350.707064533" watchObservedRunningTime="2026-10-30 21:48:24.934817946 +0000 UTC m=+350.708186434"
Oct 30 21:48:35.823384 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:48:35.823356    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-74dfcfb894-scrsv"
Oct 30 21:48:38.356108 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:48:38.356075    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-scrsv"]
Oct 30 21:48:38.356507 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:48:38.356271    4473 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-74dfcfb894-scrsv" podUID="4fa01c3e-1438-4945-9353-d5058935606e" containerName="limitador" containerID="cri-o://32f49ebd52897363aa15fccbc59172d190e39f74ab6fa6156e2977d1c28fab76" gracePeriod=30
Oct 30 21:48:38.878535 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:48:38.878504    4473 generic.go:356] "Generic (PLEG): container finished" podID="4fa01c3e-1438-4945-9353-d5058935606e" containerID="32f49ebd52897363aa15fccbc59172d190e39f74ab6fa6156e2977d1c28fab76" exitCode=0
Oct 30 21:48:38.878696 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:48:38.878581    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-74dfcfb894-scrsv" event={"ID":"4fa01c3e-1438-4945-9353-d5058935606e","Type":"ContainerDied","Data":"32f49ebd52897363aa15fccbc59172d190e39f74ab6fa6156e2977d1c28fab76"}
Oct 30 21:48:38.896537 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:48:38.896507    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-74dfcfb894-scrsv"
Oct 30 21:48:38.952466 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:48:38.952376    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4fa01c3e-1438-4945-9353-d5058935606e-kube-api-access-ghx6k\" (UniqueName: \"kubernetes.io/projected/4fa01c3e-1438-4945-9353-d5058935606e-kube-api-access-ghx6k\") pod \"4fa01c3e-1438-4945-9353-d5058935606e\" (UID: \"4fa01c3e-1438-4945-9353-d5058935606e\") "
Oct 30 21:48:38.952816 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:48:38.952433    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4fa01c3e-1438-4945-9353-d5058935606e-config-file\" (UniqueName: \"kubernetes.io/configmap/4fa01c3e-1438-4945-9353-d5058935606e-config-file\") pod \"4fa01c3e-1438-4945-9353-d5058935606e\" (UID: \"4fa01c3e-1438-4945-9353-d5058935606e\") "
Oct 30 21:48:38.953244 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:48:38.953205    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4fa01c3e-1438-4945-9353-d5058935606e-config-file" pod "4fa01c3e-1438-4945-9353-d5058935606e" (UID: "4fa01c3e-1438-4945-9353-d5058935606e"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:48:38.953595 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:48:38.953556    4473 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/4fa01c3e-1438-4945-9353-d5058935606e-config-file\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:48:38.958514 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:48:38.958470    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4fa01c3e-1438-4945-9353-d5058935606e-kube-api-access-ghx6k" pod "4fa01c3e-1438-4945-9353-d5058935606e" (UID: "4fa01c3e-1438-4945-9353-d5058935606e"). InnerVolumeSpecName "kube-api-access-ghx6k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:48:39.054801 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:48:39.054769    4473 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ghx6k\" (UniqueName: \"kubernetes.io/projected/4fa01c3e-1438-4945-9353-d5058935606e-kube-api-access-ghx6k\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:48:39.884318 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:48:39.884293    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-74dfcfb894-scrsv"
Oct 30 21:48:39.884833 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:48:39.884290    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-74dfcfb894-scrsv" event={"ID":"4fa01c3e-1438-4945-9353-d5058935606e","Type":"ContainerDied","Data":"835f8c4f34d5a412aaaf2ed0695f00c8aaab51ea35fd158709728da71ba8284b"}
Oct 30 21:48:39.884833 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:48:39.884420    4473 scope.go:122] "RemoveContainer" containerID="32f49ebd52897363aa15fccbc59172d190e39f74ab6fa6156e2977d1c28fab76"
Oct 30 21:48:40.057014 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:48:40.056982    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-scrsv"]
Oct 30 21:48:40.106983 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:48:40.106951    4473 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-scrsv"]
Oct 30 21:48:40.354912 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:48:40.354835    4473 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4fa01c3e-1438-4945-9353-d5058935606e" path="/var/lib/kubelet/pods/4fa01c3e-1438-4945-9353-d5058935606e/volumes"
Oct 30 21:48:42.949286 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:48:42.949246    4473 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-7cdbf6dc44-h5d9q"]
Oct 30 21:48:42.949771 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:48:42.949685    4473 cpu_manager.go:414] "containerMap: removing container" podUID="4fa01c3e-1438-4945-9353-d5058935606e" containerName="limitador"
Oct 30 21:48:42.949771 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:48:42.949703    4473 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4fa01c3e-1438-4945-9353-d5058935606e" containerName="limitador"
Oct 30 21:48:42.949878 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:48:42.949791    4473 memory_manager.go:365] "RemoveStaleState removing state" podUID="4fa01c3e-1438-4945-9353-d5058935606e" containerName="limitador"
Oct 30 21:48:42.954217 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:48:42.954194    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-h5d9q"
Oct 30 21:48:42.989536 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:48:42.989508    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7v6tw\" (UniqueName: \"kubernetes.io/projected/e5b0a336-6057-4b8e-8803-526f1398d0f8-kube-api-access-7v6tw\") pod \"postgres-7cdbf6dc44-h5d9q\" (UID: \"e5b0a336-6057-4b8e-8803-526f1398d0f8\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-h5d9q"
Oct 30 21:48:42.989688 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:48:42.989568    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/e5b0a336-6057-4b8e-8803-526f1398d0f8-data\") pod \"postgres-7cdbf6dc44-h5d9q\" (UID: \"e5b0a336-6057-4b8e-8803-526f1398d0f8\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-h5d9q"
Oct 30 21:48:43.008777 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:48:43.008752    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Oct 30 21:48:43.058195 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:48:43.058163    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-j5gh4\""
Oct 30 21:48:43.058195 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:48:43.058166    4473 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 30 21:48:43.058392 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:48:43.058293    4473 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 30 21:48:43.059406 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:48:43.059386    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-7cdbf6dc44-h5d9q"]
Oct 30 21:48:43.090391 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:48:43.090356    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7v6tw\" (UniqueName: \"kubernetes.io/projected/e5b0a336-6057-4b8e-8803-526f1398d0f8-kube-api-access-7v6tw\") pod \"postgres-7cdbf6dc44-h5d9q\" (UID: \"e5b0a336-6057-4b8e-8803-526f1398d0f8\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-h5d9q"
Oct 30 21:48:43.090555 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:48:43.090413    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/e5b0a336-6057-4b8e-8803-526f1398d0f8-data\") pod \"postgres-7cdbf6dc44-h5d9q\" (UID: \"e5b0a336-6057-4b8e-8803-526f1398d0f8\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-h5d9q"
Oct 30 21:48:43.090821 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:48:43.090804    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/e5b0a336-6057-4b8e-8803-526f1398d0f8-data\") pod \"postgres-7cdbf6dc44-h5d9q\" (UID: \"e5b0a336-6057-4b8e-8803-526f1398d0f8\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-h5d9q"
Oct 30 21:48:43.150571 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:48:43.150533    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7v6tw\" (UniqueName: \"kubernetes.io/projected/e5b0a336-6057-4b8e-8803-526f1398d0f8-kube-api-access-7v6tw\") pod \"postgres-7cdbf6dc44-h5d9q\" (UID: \"e5b0a336-6057-4b8e-8803-526f1398d0f8\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-h5d9q"
Oct 30 21:48:43.266585 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:48:43.266503    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-h5d9q"
Oct 30 21:48:43.633621 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:48:43.633599    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-7cdbf6dc44-h5d9q"]
Oct 30 21:48:43.635863 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:48:43.635819    4473 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5b0a336_6057_4b8e_8803_526f1398d0f8.slice/crio-5c43026f92bfb1a85f8a73c11cac6a3c0a9e2dbc13c2a599e9d56ca20b4e3d73 WatchSource:0}: Error finding container 5c43026f92bfb1a85f8a73c11cac6a3c0a9e2dbc13c2a599e9d56ca20b4e3d73: Status 404 returned error can't find the container with id 5c43026f92bfb1a85f8a73c11cac6a3c0a9e2dbc13c2a599e9d56ca20b4e3d73
Oct 30 21:48:43.880325 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:48:43.880295    4473 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-module-controller-manager-7566cf5fc8-8l7qq"]
Oct 30 21:48:43.884965 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:48:43.884873    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-8l7qq"
Oct 30 21:48:43.898210 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:48:43.898189    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9f6d6982-07b1-44d1-a126-27c73898a66e-tmp\") pod \"kserve-module-controller-manager-7566cf5fc8-8l7qq\" (UID: \"9f6d6982-07b1-44d1-a126-27c73898a66e\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-8l7qq"
Oct 30 21:48:43.898317 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:48:43.898227    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q4hxj\" (UniqueName: \"kubernetes.io/projected/9f6d6982-07b1-44d1-a126-27c73898a66e-kube-api-access-q4hxj\") pod \"kserve-module-controller-manager-7566cf5fc8-8l7qq\" (UID: \"9f6d6982-07b1-44d1-a126-27c73898a66e\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-8l7qq"
Oct 30 21:48:43.898317 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:48:43.898295    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/9f6d6982-07b1-44d1-a126-27c73898a66e-manifests\") pod \"kserve-module-controller-manager-7566cf5fc8-8l7qq\" (UID: \"9f6d6982-07b1-44d1-a126-27c73898a66e\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-8l7qq"
Oct 30 21:48:43.898421 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:48:43.898329    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9f6d6982-07b1-44d1-a126-27c73898a66e-trusted-ca\") pod \"kserve-module-controller-manager-7566cf5fc8-8l7qq\" (UID: \"9f6d6982-07b1-44d1-a126-27c73898a66e\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-8l7qq"
Oct 30 21:48:43.901545 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:48:43.901524    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-h5d9q" event={"ID":"e5b0a336-6057-4b8e-8803-526f1398d0f8","Type":"ContainerStarted","Data":"5c43026f92bfb1a85f8a73c11cac6a3c0a9e2dbc13c2a599e9d56ca20b4e3d73"}
Oct 30 21:48:43.943246 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:48:43.943224    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-module-controller-manager-dockercfg-97wwf\""
Oct 30 21:48:43.995177 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:48:43.995144    4473 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"odh-trusted-ca-bundle\""
Oct 30 21:48:43.996299 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:48:43.996278    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-module-controller-manager-7566cf5fc8-8l7qq"]
Oct 30 21:48:43.998978 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:48:43.998960    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9f6d6982-07b1-44d1-a126-27c73898a66e-tmp\") pod \"kserve-module-controller-manager-7566cf5fc8-8l7qq\" (UID: \"9f6d6982-07b1-44d1-a126-27c73898a66e\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-8l7qq"
Oct 30 21:48:43.999027 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:48:43.998997    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q4hxj\" (UniqueName: \"kubernetes.io/projected/9f6d6982-07b1-44d1-a126-27c73898a66e-kube-api-access-q4hxj\") pod \"kserve-module-controller-manager-7566cf5fc8-8l7qq\" (UID: \"9f6d6982-07b1-44d1-a126-27c73898a66e\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-8l7qq"
Oct 30 21:48:43.999027 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:48:43.999020    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/9f6d6982-07b1-44d1-a126-27c73898a66e-manifests\") pod \"kserve-module-controller-manager-7566cf5fc8-8l7qq\" (UID: \"9f6d6982-07b1-44d1-a126-27c73898a66e\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-8l7qq"
Oct 30 21:48:43.999102 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:48:43.999052    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9f6d6982-07b1-44d1-a126-27c73898a66e-trusted-ca\") pod \"kserve-module-controller-manager-7566cf5fc8-8l7qq\" (UID: \"9f6d6982-07b1-44d1-a126-27c73898a66e\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-8l7qq"
Oct 30 21:48:43.999342 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:48:43.999327    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9f6d6982-07b1-44d1-a126-27c73898a66e-tmp\") pod \"kserve-module-controller-manager-7566cf5fc8-8l7qq\" (UID: \"9f6d6982-07b1-44d1-a126-27c73898a66e\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-8l7qq"
Oct 30 21:48:43.999389 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:48:43.999371    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/9f6d6982-07b1-44d1-a126-27c73898a66e-manifests\") pod \"kserve-module-controller-manager-7566cf5fc8-8l7qq\" (UID: \"9f6d6982-07b1-44d1-a126-27c73898a66e\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-8l7qq"
Oct 30 21:48:43.999891 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:48:43.999875    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9f6d6982-07b1-44d1-a126-27c73898a66e-trusted-ca\") pod \"kserve-module-controller-manager-7566cf5fc8-8l7qq\" (UID: \"9f6d6982-07b1-44d1-a126-27c73898a66e\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-8l7qq"
Oct 30 21:48:44.042735 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:48:44.042707    4473 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Oct 30 21:48:44.042735 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:48:44.042710    4473 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Oct 30 21:48:44.057302 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:48:44.057270    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q4hxj\" (UniqueName: \"kubernetes.io/projected/9f6d6982-07b1-44d1-a126-27c73898a66e-kube-api-access-q4hxj\") pod \"kserve-module-controller-manager-7566cf5fc8-8l7qq\" (UID: \"9f6d6982-07b1-44d1-a126-27c73898a66e\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-8l7qq"
Oct 30 21:48:44.197078 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:48:44.196998    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-8l7qq"
Oct 30 21:48:44.618145 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:48:44.618121    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-module-controller-manager-7566cf5fc8-8l7qq"]
Oct 30 21:48:44.619997 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:48:44.619970    4473 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f6d6982_07b1_44d1_a126_27c73898a66e.slice/crio-742e423a5b2b3b802231bb76b2be02375b60335df7739168b67373c7c39e5cec WatchSource:0}: Error finding container 742e423a5b2b3b802231bb76b2be02375b60335df7739168b67373c7c39e5cec: Status 404 returned error can't find the container with id 742e423a5b2b3b802231bb76b2be02375b60335df7739168b67373c7c39e5cec
Oct 30 21:48:44.906480 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:48:44.906399    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-8l7qq" event={"ID":"9f6d6982-07b1-44d1-a126-27c73898a66e","Type":"ContainerStarted","Data":"742e423a5b2b3b802231bb76b2be02375b60335df7739168b67373c7c39e5cec"}
Oct 30 21:48:46.918375 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:48:46.918336    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-8l7qq" event={"ID":"9f6d6982-07b1-44d1-a126-27c73898a66e","Type":"ContainerStarted","Data":"a54be8327f03bdb4f951fd76f18f0287fdcea31f126bc6807161332f32265844"}
Oct 30 21:48:46.918794 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:48:46.918442    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-8l7qq"
Oct 30 21:48:47.053896 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:48:47.053792    4473 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-8l7qq" podStartSLOduration=1.908218255 podStartE2EDuration="4.053777787s" totalImagesPullingTime="2.145559532s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:48:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:48:47.05308199 +0000 UTC m=+372.826450474" watchObservedRunningTime="2026-10-30 21:48:47.053777787 +0000 UTC m=+372.827146300"
Oct 30 21:48:50.322858 ip-10-0-140-18 kubenswrapper[4473]: E1030 21:48:50.322823    4473 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fncvg" containerName="extractor"
Oct 30 21:48:50.938400 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:48:50.938357    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-h5d9q" event={"ID":"e5b0a336-6057-4b8e-8803-526f1398d0f8","Type":"ContainerStarted","Data":"8a3f293b29d22d18f9a14385ece67e6741161c869cc51f46b4424ea1db62f89b"}
Oct 30 21:48:50.938593 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:48:50.938501    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-h5d9q"
Oct 30 21:48:51.053652 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:48:51.053598    4473 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-h5d9q" podStartSLOduration=2.499111422 podStartE2EDuration="9.053585105s" totalImagesPullingTime="6.554473683s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:48:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:48:51.05289315 +0000 UTC m=+376.826261637" watchObservedRunningTime="2026-10-30 21:48:51.053585105 +0000 UTC m=+376.826953590"
Oct 30 21:48:56.970605 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:48:56.970579    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-h5d9q"
Oct 30 21:49:03.713551 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:03.713512    4473 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-579cf7c6d-jq2bt"]
Oct 30 21:49:03.721281 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:03.721256    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-579cf7c6d-jq2bt"
Oct 30 21:49:03.772241 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:03.772210    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f82q7\" (UniqueName: \"kubernetes.io/projected/b084a28b-9d98-4c68-9020-b38d6464262f-kube-api-access-f82q7\") pod \"maas-controller-579cf7c6d-jq2bt\" (UID: \"b084a28b-9d98-4c68-9020-b38d6464262f\") " pod="opendatahub/maas-controller-579cf7c6d-jq2bt"
Oct 30 21:49:03.772425 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:03.772263    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b084a28b-9d98-4c68-9020-b38d6464262f-webhook-cert\") pod \"maas-controller-579cf7c6d-jq2bt\" (UID: \"b084a28b-9d98-4c68-9020-b38d6464262f\") " pod="opendatahub/maas-controller-579cf7c6d-jq2bt"
Oct 30 21:49:03.772425 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:03.772369    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b084a28b-9d98-4c68-9020-b38d6464262f-metrics-certs\") pod \"maas-controller-579cf7c6d-jq2bt\" (UID: \"b084a28b-9d98-4c68-9020-b38d6464262f\") " pod="opendatahub/maas-controller-579cf7c6d-jq2bt"
Oct 30 21:49:03.774554 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:03.774513    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Oct 30 21:49:03.774554 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:03.774521    4473 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Oct 30 21:49:03.804439 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:03.804412    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Oct 30 21:49:03.824890 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:03.824852    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-qcnnj\""
Oct 30 21:49:03.826177 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:03.826153    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-579cf7c6d-jq2bt"]
Oct 30 21:49:03.873156 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:03.873125    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b084a28b-9d98-4c68-9020-b38d6464262f-webhook-cert\") pod \"maas-controller-579cf7c6d-jq2bt\" (UID: \"b084a28b-9d98-4c68-9020-b38d6464262f\") " pod="opendatahub/maas-controller-579cf7c6d-jq2bt"
Oct 30 21:49:03.873323 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:03.873207    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b084a28b-9d98-4c68-9020-b38d6464262f-metrics-certs\") pod \"maas-controller-579cf7c6d-jq2bt\" (UID: \"b084a28b-9d98-4c68-9020-b38d6464262f\") " pod="opendatahub/maas-controller-579cf7c6d-jq2bt"
Oct 30 21:49:03.873323 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:03.873272    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f82q7\" (UniqueName: \"kubernetes.io/projected/b084a28b-9d98-4c68-9020-b38d6464262f-kube-api-access-f82q7\") pod \"maas-controller-579cf7c6d-jq2bt\" (UID: \"b084a28b-9d98-4c68-9020-b38d6464262f\") " pod="opendatahub/maas-controller-579cf7c6d-jq2bt"
Oct 30 21:49:03.875999 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:03.875975    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b084a28b-9d98-4c68-9020-b38d6464262f-metrics-certs\") pod \"maas-controller-579cf7c6d-jq2bt\" (UID: \"b084a28b-9d98-4c68-9020-b38d6464262f\") " pod="opendatahub/maas-controller-579cf7c6d-jq2bt"
Oct 30 21:49:03.876117 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:03.875980    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b084a28b-9d98-4c68-9020-b38d6464262f-webhook-cert\") pod \"maas-controller-579cf7c6d-jq2bt\" (UID: \"b084a28b-9d98-4c68-9020-b38d6464262f\") " pod="opendatahub/maas-controller-579cf7c6d-jq2bt"
Oct 30 21:49:03.932238 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:03.932209    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f82q7\" (UniqueName: \"kubernetes.io/projected/b084a28b-9d98-4c68-9020-b38d6464262f-kube-api-access-f82q7\") pod \"maas-controller-579cf7c6d-jq2bt\" (UID: \"b084a28b-9d98-4c68-9020-b38d6464262f\") " pod="opendatahub/maas-controller-579cf7c6d-jq2bt"
Oct 30 21:49:04.034436 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:04.034348    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-579cf7c6d-jq2bt"
Oct 30 21:49:04.106904 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:04.106071    4473 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-786df48447-m2d88"]
Oct 30 21:49:04.111099 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:04.111075    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-786df48447-m2d88"
Oct 30 21:49:04.167949 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:04.167905    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-786df48447-m2d88"]
Oct 30 21:49:04.175868 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:04.175842    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e5ce0eb3-a29d-4c95-b72f-a84a43f98ea9-metrics-certs\") pod \"maas-controller-786df48447-m2d88\" (UID: \"e5ce0eb3-a29d-4c95-b72f-a84a43f98ea9\") " pod="opendatahub/maas-controller-786df48447-m2d88"
Oct 30 21:49:04.176034 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:04.175884    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e5ce0eb3-a29d-4c95-b72f-a84a43f98ea9-webhook-cert\") pod \"maas-controller-786df48447-m2d88\" (UID: \"e5ce0eb3-a29d-4c95-b72f-a84a43f98ea9\") " pod="opendatahub/maas-controller-786df48447-m2d88"
Oct 30 21:49:04.176034 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:04.175996    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cx8p4\" (UniqueName: \"kubernetes.io/projected/e5ce0eb3-a29d-4c95-b72f-a84a43f98ea9-kube-api-access-cx8p4\") pod \"maas-controller-786df48447-m2d88\" (UID: \"e5ce0eb3-a29d-4c95-b72f-a84a43f98ea9\") " pod="opendatahub/maas-controller-786df48447-m2d88"
Oct 30 21:49:04.277160 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:04.277128    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e5ce0eb3-a29d-4c95-b72f-a84a43f98ea9-webhook-cert\") pod \"maas-controller-786df48447-m2d88\" (UID: \"e5ce0eb3-a29d-4c95-b72f-a84a43f98ea9\") " pod="opendatahub/maas-controller-786df48447-m2d88"
Oct 30 21:49:04.277325 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:04.277175    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cx8p4\" (UniqueName: \"kubernetes.io/projected/e5ce0eb3-a29d-4c95-b72f-a84a43f98ea9-kube-api-access-cx8p4\") pod \"maas-controller-786df48447-m2d88\" (UID: \"e5ce0eb3-a29d-4c95-b72f-a84a43f98ea9\") " pod="opendatahub/maas-controller-786df48447-m2d88"
Oct 30 21:49:04.277325 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:04.277219    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e5ce0eb3-a29d-4c95-b72f-a84a43f98ea9-metrics-certs\") pod \"maas-controller-786df48447-m2d88\" (UID: \"e5ce0eb3-a29d-4c95-b72f-a84a43f98ea9\") " pod="opendatahub/maas-controller-786df48447-m2d88"
Oct 30 21:49:04.279779 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:04.279755    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e5ce0eb3-a29d-4c95-b72f-a84a43f98ea9-metrics-certs\") pod \"maas-controller-786df48447-m2d88\" (UID: \"e5ce0eb3-a29d-4c95-b72f-a84a43f98ea9\") " pod="opendatahub/maas-controller-786df48447-m2d88"
Oct 30 21:49:04.279898 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:04.279786    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e5ce0eb3-a29d-4c95-b72f-a84a43f98ea9-webhook-cert\") pod \"maas-controller-786df48447-m2d88\" (UID: \"e5ce0eb3-a29d-4c95-b72f-a84a43f98ea9\") " pod="opendatahub/maas-controller-786df48447-m2d88"
Oct 30 21:49:04.334636 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:04.334566    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cx8p4\" (UniqueName: \"kubernetes.io/projected/e5ce0eb3-a29d-4c95-b72f-a84a43f98ea9-kube-api-access-cx8p4\") pod \"maas-controller-786df48447-m2d88\" (UID: \"e5ce0eb3-a29d-4c95-b72f-a84a43f98ea9\") " pod="opendatahub/maas-controller-786df48447-m2d88"
Oct 30 21:49:04.402787 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:04.402766    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-579cf7c6d-jq2bt"]
Oct 30 21:49:04.404698 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:49:04.404667    4473 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb084a28b_9d98_4c68_9020_b38d6464262f.slice/crio-6f96f40b5e6ca9839237d6cc458f1d1633e131193b6024e33c10de5f028e86d8 WatchSource:0}: Error finding container 6f96f40b5e6ca9839237d6cc458f1d1633e131193b6024e33c10de5f028e86d8: Status 404 returned error can't find the container with id 6f96f40b5e6ca9839237d6cc458f1d1633e131193b6024e33c10de5f028e86d8
Oct 30 21:49:04.424763 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:04.424717    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-786df48447-m2d88"
Oct 30 21:49:04.796054 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:04.796032    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-786df48447-m2d88"]
Oct 30 21:49:04.797523 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:49:04.797486    4473 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5ce0eb3_a29d_4c95_b72f_a84a43f98ea9.slice/crio-ba1078d24136e40ae6f639cebe784c9b0a90384cadb4440246fdfaa5cdd8d33d WatchSource:0}: Error finding container ba1078d24136e40ae6f639cebe784c9b0a90384cadb4440246fdfaa5cdd8d33d: Status 404 returned error can't find the container with id ba1078d24136e40ae6f639cebe784c9b0a90384cadb4440246fdfaa5cdd8d33d
Oct 30 21:49:04.999423 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:04.999385    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-786df48447-m2d88" event={"ID":"e5ce0eb3-a29d-4c95-b72f-a84a43f98ea9","Type":"ContainerStarted","Data":"ba1078d24136e40ae6f639cebe784c9b0a90384cadb4440246fdfaa5cdd8d33d"}
Oct 30 21:49:05.000417 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:05.000396    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-579cf7c6d-jq2bt" event={"ID":"b084a28b-9d98-4c68-9020-b38d6464262f","Type":"ContainerStarted","Data":"6f96f40b5e6ca9839237d6cc458f1d1633e131193b6024e33c10de5f028e86d8"}
Oct 30 21:49:07.924379 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:07.924357    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-8l7qq"
Oct 30 21:49:08.019232 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:08.019204    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-579cf7c6d-jq2bt" event={"ID":"b084a28b-9d98-4c68-9020-b38d6464262f","Type":"ContainerStarted","Data":"a6488933f908195531af6e26e1799eb51106ec021894c44b4b51c2c35f5d026d"}
Oct 30 21:49:08.019461 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:08.019308    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-579cf7c6d-jq2bt"
Oct 30 21:49:08.204255 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:08.204161    4473 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/maas-controller-579cf7c6d-jq2bt" podStartSLOduration=1.696480331 podStartE2EDuration="5.204145101s" totalImagesPullingTime="3.50766477s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:49:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:49:08.203206014 +0000 UTC m=+393.976574500" watchObservedRunningTime="2026-10-30 21:49:08.204145101 +0000 UTC m=+393.977513581"
Oct 30 21:49:09.025455 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:09.025424    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-786df48447-m2d88" event={"ID":"e5ce0eb3-a29d-4c95-b72f-a84a43f98ea9","Type":"ContainerStarted","Data":"17b0eb34ebff9edb126f961c4de1126619cd3675f50d16ecd274e447fe7b0410"}
Oct 30 21:49:09.025904 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:09.025528    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-786df48447-m2d88"
Oct 30 21:49:09.142345 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:09.142283    4473 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/maas-controller-786df48447-m2d88" podStartSLOduration=1.960576308 podStartE2EDuration="5.142265169s" totalImagesPullingTime="3.181688861s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:49:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:49:09.141522427 +0000 UTC m=+394.914890912" watchObservedRunningTime="2026-10-30 21:49:09.142265169 +0000 UTC m=+394.915633656"
Oct 30 21:49:11.252439 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:11.252399    4473 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-9cgjz"]
Oct 30 21:49:11.256341 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:11.256315    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-9cgjz"
Oct 30 21:49:11.349354 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:11.349318    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-source\" (UniqueName: \"kubernetes.io/configmap/8ce72a54-9bb1-47c9-9891-3e998dcf8e2e-proxy-source\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-9cgjz\" (UID: \"8ce72a54-9bb1-47c9-9891-3e998dcf8e2e\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-9cgjz"
Oct 30 21:49:11.349529 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:11.349362    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8ce72a54-9bb1-47c9-9891-3e998dcf8e2e-service-ca\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-9cgjz\" (UID: \"8ce72a54-9bb1-47c9-9891-3e998dcf8e2e\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-9cgjz"
Oct 30 21:49:11.349529 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:11.349426    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/8ce72a54-9bb1-47c9-9891-3e998dcf8e2e-tls-cert\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-9cgjz\" (UID: \"8ce72a54-9bb1-47c9-9891-3e998dcf8e2e\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-9cgjz"
Oct 30 21:49:11.349643 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:11.349524    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g92cr\" (UniqueName: \"kubernetes.io/projected/8ce72a54-9bb1-47c9-9891-3e998dcf8e2e-kube-api-access-g92cr\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-9cgjz\" (UID: \"8ce72a54-9bb1-47c9-9891-3e998dcf8e2e\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-9cgjz"
Oct 30 21:49:11.361713 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:11.361680    4473 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"usage-logs-tenancy-proxy-ca-bundle\""
Oct 30 21:49:11.361867 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:11.361736    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"usage-logs-tenancy-proxy-tls\""
Oct 30 21:49:11.361867 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:11.361776    4473 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"usage-tenancy-proxy-source\""
Oct 30 21:49:11.361980 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:11.361880    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"usage-tenancy-proxy-dockercfg-b6mns\""
Oct 30 21:49:11.363291 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:11.363266    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-9cgjz"]
Oct 30 21:49:11.450650 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:11.450606    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g92cr\" (UniqueName: \"kubernetes.io/projected/8ce72a54-9bb1-47c9-9891-3e998dcf8e2e-kube-api-access-g92cr\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-9cgjz\" (UID: \"8ce72a54-9bb1-47c9-9891-3e998dcf8e2e\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-9cgjz"
Oct 30 21:49:11.450851 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:11.450748    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-source\" (UniqueName: \"kubernetes.io/configmap/8ce72a54-9bb1-47c9-9891-3e998dcf8e2e-proxy-source\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-9cgjz\" (UID: \"8ce72a54-9bb1-47c9-9891-3e998dcf8e2e\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-9cgjz"
Oct 30 21:49:11.450851 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:11.450767    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8ce72a54-9bb1-47c9-9891-3e998dcf8e2e-service-ca\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-9cgjz\" (UID: \"8ce72a54-9bb1-47c9-9891-3e998dcf8e2e\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-9cgjz"
Oct 30 21:49:11.450851 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:11.450785    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/8ce72a54-9bb1-47c9-9891-3e998dcf8e2e-tls-cert\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-9cgjz\" (UID: \"8ce72a54-9bb1-47c9-9891-3e998dcf8e2e\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-9cgjz"
Oct 30 21:49:11.451052 ip-10-0-140-18 kubenswrapper[4473]: E1030 21:49:11.450850    4473 configmap.go:193] Couldn't get configMap opendatahub/usage-tenancy-proxy-source: configmap "usage-tenancy-proxy-source" not found
Oct 30 21:49:11.451052 ip-10-0-140-18 kubenswrapper[4473]: E1030 21:49:11.450952    4473 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8ce72a54-9bb1-47c9-9891-3e998dcf8e2e-proxy-source podName:8ce72a54-9bb1-47c9-9891-3e998dcf8e2e nodeName:}" failed. No retries permitted until 2026-10-30 21:49:11.950914346 +0000 UTC m=+397.724282828 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-source" (UniqueName: "kubernetes.io/configmap/8ce72a54-9bb1-47c9-9891-3e998dcf8e2e-proxy-source") pod "usage-logs-tenancy-proxy-9c5d49bcd-9cgjz" (UID: "8ce72a54-9bb1-47c9-9891-3e998dcf8e2e") : configmap "usage-tenancy-proxy-source" not found
Oct 30 21:49:11.451052 ip-10-0-140-18 kubenswrapper[4473]: E1030 21:49:11.450952    4473 secret.go:189] Couldn't get secret opendatahub/usage-logs-tenancy-proxy-tls: secret "usage-logs-tenancy-proxy-tls" not found
Oct 30 21:49:11.451052 ip-10-0-140-18 kubenswrapper[4473]: E1030 21:49:11.451009    4473 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8ce72a54-9bb1-47c9-9891-3e998dcf8e2e-tls-cert podName:8ce72a54-9bb1-47c9-9891-3e998dcf8e2e nodeName:}" failed. No retries permitted until 2026-10-30 21:49:11.950997662 +0000 UTC m=+397.724366140 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-cert" (UniqueName: "kubernetes.io/secret/8ce72a54-9bb1-47c9-9891-3e998dcf8e2e-tls-cert") pod "usage-logs-tenancy-proxy-9c5d49bcd-9cgjz" (UID: "8ce72a54-9bb1-47c9-9891-3e998dcf8e2e") : secret "usage-logs-tenancy-proxy-tls" not found
Oct 30 21:49:11.451396 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:11.451377    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8ce72a54-9bb1-47c9-9891-3e998dcf8e2e-service-ca\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-9cgjz\" (UID: \"8ce72a54-9bb1-47c9-9891-3e998dcf8e2e\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-9cgjz"
Oct 30 21:49:11.509113 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:11.509028    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g92cr\" (UniqueName: \"kubernetes.io/projected/8ce72a54-9bb1-47c9-9891-3e998dcf8e2e-kube-api-access-g92cr\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-9cgjz\" (UID: \"8ce72a54-9bb1-47c9-9891-3e998dcf8e2e\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-9cgjz"
Oct 30 21:49:11.956606 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:11.956573    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-source\" (UniqueName: \"kubernetes.io/configmap/8ce72a54-9bb1-47c9-9891-3e998dcf8e2e-proxy-source\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-9cgjz\" (UID: \"8ce72a54-9bb1-47c9-9891-3e998dcf8e2e\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-9cgjz"
Oct 30 21:49:11.956606 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:11.956611    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/8ce72a54-9bb1-47c9-9891-3e998dcf8e2e-tls-cert\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-9cgjz\" (UID: \"8ce72a54-9bb1-47c9-9891-3e998dcf8e2e\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-9cgjz"
Oct 30 21:49:11.957469 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:11.957445    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-source\" (UniqueName: \"kubernetes.io/configmap/8ce72a54-9bb1-47c9-9891-3e998dcf8e2e-proxy-source\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-9cgjz\" (UID: \"8ce72a54-9bb1-47c9-9891-3e998dcf8e2e\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-9cgjz"
Oct 30 21:49:11.959344 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:11.959321    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/8ce72a54-9bb1-47c9-9891-3e998dcf8e2e-tls-cert\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-9cgjz\" (UID: \"8ce72a54-9bb1-47c9-9891-3e998dcf8e2e\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-9cgjz"
Oct 30 21:49:12.168743 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:12.168705    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-9cgjz"
Oct 30 21:49:12.539950 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:12.539909    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-9cgjz"]
Oct 30 21:49:12.541909 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:49:12.541885    4473 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ce72a54_9bb1_47c9_9891_3e998dcf8e2e.slice/crio-d90200aae447a6235e6a1495b09a2ac330de90e09d4adf8b86a877ca4dbe0b7f WatchSource:0}: Error finding container d90200aae447a6235e6a1495b09a2ac330de90e09d4adf8b86a877ca4dbe0b7f: Status 404 returned error can't find the container with id d90200aae447a6235e6a1495b09a2ac330de90e09d4adf8b86a877ca4dbe0b7f
Oct 30 21:49:13.044505 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:13.044468    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-9cgjz" event={"ID":"8ce72a54-9bb1-47c9-9891-3e998dcf8e2e","Type":"ContainerStarted","Data":"d90200aae447a6235e6a1495b09a2ac330de90e09d4adf8b86a877ca4dbe0b7f"}
Oct 30 21:49:19.029372 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:19.028291    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-579cf7c6d-jq2bt"
Oct 30 21:49:20.033940 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:20.033881    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-786df48447-m2d88"
Oct 30 21:49:20.206776 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:20.206745    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-579cf7c6d-jq2bt"]
Oct 30 21:49:20.207035 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:20.207011    4473 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/maas-controller-579cf7c6d-jq2bt" podUID="b084a28b-9d98-4c68-9020-b38d6464262f" containerName="manager" containerID="cri-o://a6488933f908195531af6e26e1799eb51106ec021894c44b4b51c2c35f5d026d" gracePeriod=30
Oct 30 21:49:20.332099 ip-10-0-140-18 kubenswrapper[4473]: E1030 21:49:20.332058    4473 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fncvg" containerName="extractor"
Oct 30 21:49:20.503517 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:20.503481    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-579cf7c6d-jq2bt"
Oct 30 21:49:20.646332 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:20.646279    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b084a28b-9d98-4c68-9020-b38d6464262f-webhook-cert\" (UniqueName: \"kubernetes.io/secret/b084a28b-9d98-4c68-9020-b38d6464262f-webhook-cert\") pod \"b084a28b-9d98-4c68-9020-b38d6464262f\" (UID: \"b084a28b-9d98-4c68-9020-b38d6464262f\") "
Oct 30 21:49:20.646519 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:20.646372    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b084a28b-9d98-4c68-9020-b38d6464262f-metrics-certs\" (UniqueName: \"kubernetes.io/secret/b084a28b-9d98-4c68-9020-b38d6464262f-metrics-certs\") pod \"b084a28b-9d98-4c68-9020-b38d6464262f\" (UID: \"b084a28b-9d98-4c68-9020-b38d6464262f\") "
Oct 30 21:49:20.646519 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:20.646418    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b084a28b-9d98-4c68-9020-b38d6464262f-kube-api-access-f82q7\" (UniqueName: \"kubernetes.io/projected/b084a28b-9d98-4c68-9020-b38d6464262f-kube-api-access-f82q7\") pod \"b084a28b-9d98-4c68-9020-b38d6464262f\" (UID: \"b084a28b-9d98-4c68-9020-b38d6464262f\") "
Oct 30 21:49:20.649686 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:20.649649    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b084a28b-9d98-4c68-9020-b38d6464262f-kube-api-access-f82q7" pod "b084a28b-9d98-4c68-9020-b38d6464262f" (UID: "b084a28b-9d98-4c68-9020-b38d6464262f"). InnerVolumeSpecName "kube-api-access-f82q7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:49:20.650308 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:20.650277    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b084a28b-9d98-4c68-9020-b38d6464262f-webhook-cert" pod "b084a28b-9d98-4c68-9020-b38d6464262f" (UID: "b084a28b-9d98-4c68-9020-b38d6464262f"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:49:20.650620 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:20.650586    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b084a28b-9d98-4c68-9020-b38d6464262f-metrics-certs" pod "b084a28b-9d98-4c68-9020-b38d6464262f" (UID: "b084a28b-9d98-4c68-9020-b38d6464262f"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:49:20.747169 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:20.747133    4473 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b084a28b-9d98-4c68-9020-b38d6464262f-webhook-cert\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:49:20.747169 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:20.747166    4473 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b084a28b-9d98-4c68-9020-b38d6464262f-metrics-certs\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:49:20.747169 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:20.747177    4473 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f82q7\" (UniqueName: \"kubernetes.io/projected/b084a28b-9d98-4c68-9020-b38d6464262f-kube-api-access-f82q7\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:49:21.086532 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:21.086437    4473 generic.go:356] "Generic (PLEG): container finished" podID="b084a28b-9d98-4c68-9020-b38d6464262f" containerID="a6488933f908195531af6e26e1799eb51106ec021894c44b4b51c2c35f5d026d" exitCode=0
Oct 30 21:49:21.086532 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:21.086486    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-579cf7c6d-jq2bt"
Oct 30 21:49:21.087052 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:21.086535    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-579cf7c6d-jq2bt" event={"ID":"b084a28b-9d98-4c68-9020-b38d6464262f","Type":"ContainerDied","Data":"a6488933f908195531af6e26e1799eb51106ec021894c44b4b51c2c35f5d026d"}
Oct 30 21:49:21.087052 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:21.086589    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-579cf7c6d-jq2bt" event={"ID":"b084a28b-9d98-4c68-9020-b38d6464262f","Type":"ContainerDied","Data":"6f96f40b5e6ca9839237d6cc458f1d1633e131193b6024e33c10de5f028e86d8"}
Oct 30 21:49:21.087052 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:21.086618    4473 scope.go:122] "RemoveContainer" containerID="a6488933f908195531af6e26e1799eb51106ec021894c44b4b51c2c35f5d026d"
Oct 30 21:49:21.098010 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:21.097980    4473 scope.go:122] "RemoveContainer" containerID="a6488933f908195531af6e26e1799eb51106ec021894c44b4b51c2c35f5d026d"
Oct 30 21:49:21.098340 ip-10-0-140-18 kubenswrapper[4473]: E1030 21:49:21.098320    4473 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a6488933f908195531af6e26e1799eb51106ec021894c44b4b51c2c35f5d026d\": container with ID starting with a6488933f908195531af6e26e1799eb51106ec021894c44b4b51c2c35f5d026d not found: ID does not exist" containerID="a6488933f908195531af6e26e1799eb51106ec021894c44b4b51c2c35f5d026d"
Oct 30 21:49:21.098422 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:21.098353    4473 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a6488933f908195531af6e26e1799eb51106ec021894c44b4b51c2c35f5d026d"} err="failed to get container status \"a6488933f908195531af6e26e1799eb51106ec021894c44b4b51c2c35f5d026d\": rpc error: code = NotFound desc = could not find container \"a6488933f908195531af6e26e1799eb51106ec021894c44b4b51c2c35f5d026d\": container with ID starting with a6488933f908195531af6e26e1799eb51106ec021894c44b4b51c2c35f5d026d not found: ID does not exist"
Oct 30 21:49:21.255695 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:21.255657    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-579cf7c6d-jq2bt"]
Oct 30 21:49:21.304817 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:21.304778    4473 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-579cf7c6d-jq2bt"]
Oct 30 21:49:22.357111 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:22.357077    4473 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b084a28b-9d98-4c68-9020-b38d6464262f" path="/var/lib/kubelet/pods/b084a28b-9d98-4c68-9020-b38d6464262f/volumes"
Oct 30 21:49:27.124535 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:27.124495    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-9cgjz" event={"ID":"8ce72a54-9bb1-47c9-9891-3e998dcf8e2e","Type":"ContainerStarted","Data":"626caaa1af2a75775bcffb521c30f265d633ef4ba5b359963343b7effcab136c"}
Oct 30 21:49:27.125028 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:27.124674    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-9cgjz"
Oct 30 21:49:27.246663 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:27.246610    4473 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-9cgjz" podStartSLOduration=2.410525432 podStartE2EDuration="16.246592462s" totalImagesPullingTime="13.83606703s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:49:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:49:27.245718311 +0000 UTC m=+413.019086792" watchObservedRunningTime="2026-10-30 21:49:27.246592462 +0000 UTC m=+413.019960949"
Oct 30 21:49:34.378245 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:34.378209    4473 scope.go:122] "RemoveContainer" containerID="f7275c39e728501f8f0bcae2c877d72e456bbf0b60fbf9fb94865572cfc832c1"
Oct 30 21:49:34.386403 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:34.386379    4473 scope.go:122] "RemoveContainer" containerID="66286a6856294a63ddfaa22d609c84b6e5906d98ae9d0ba09b874c27ff13297c"
Oct 30 21:49:34.393946 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:34.393902    4473 scope.go:122] "RemoveContainer" containerID="afa044491852d471fe3b0988530c11346b561a06fad61748858586958c55bbc9"
Oct 30 21:49:34.401519 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:34.401498    4473 scope.go:122] "RemoveContainer" containerID="6b0b1baeba17b4e095f141ee41ad011f23b1b46f56082ebdc1bc1fadf1809b21"
Oct 30 21:49:34.409692 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:34.409670    4473 scope.go:122] "RemoveContainer" containerID="afc8e513226b883d9f14b81c0a39e22958c330aa231326dd8ca39c9b1e1ee351"
Oct 30 21:49:34.418419 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:34.418399    4473 scope.go:122] "RemoveContainer" containerID="e8563f4e66dc3881af914b74b03c357d9849ce9b67ed7ea889e472ca524230e7"
Oct 30 21:49:34.426304 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:34.426283    4473 scope.go:122] "RemoveContainer" containerID="ae900c17c87ccc62c744b8d1eee0f2733f1768070492e8b7285c88217918f065"
Oct 30 21:49:38.136147 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:38.136116    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-9cgjz"
Oct 30 21:49:44.891554 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:44.891516    4473 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-6d669c7fcd-gn8v4"]
Oct 30 21:49:44.891965 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:44.891841    4473 cpu_manager.go:414] "containerMap: removing container" podUID="b084a28b-9d98-4c68-9020-b38d6464262f" containerName="manager"
Oct 30 21:49:44.891965 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:44.891852    4473 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b084a28b-9d98-4c68-9020-b38d6464262f" containerName="manager"
Oct 30 21:49:44.891965 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:44.891956    4473 memory_manager.go:365] "RemoveStaleState removing state" podUID="b084a28b-9d98-4c68-9020-b38d6464262f" containerName="manager"
Oct 30 21:49:44.899265 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:44.899237    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-gn8v4"
Oct 30 21:49:44.943336 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:44.943295    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f587fb39-646c-4b6b-8451-4713172f35b0-metrics-certs\") pod \"maas-api-6d669c7fcd-gn8v4\" (UID: \"f587fb39-646c-4b6b-8451-4713172f35b0\") " pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-gn8v4"
Oct 30 21:49:44.943336 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:44.943342    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n8k58\" (UniqueName: \"kubernetes.io/projected/f587fb39-646c-4b6b-8451-4713172f35b0-kube-api-access-n8k58\") pod \"maas-api-6d669c7fcd-gn8v4\" (UID: \"f587fb39-646c-4b6b-8451-4713172f35b0\") " pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-gn8v4"
Oct 30 21:49:44.943596 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:44.943455    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f587fb39-646c-4b6b-8451-4713172f35b0-maas-api-tls\") pod \"maas-api-6d669c7fcd-gn8v4\" (UID: \"f587fb39-646c-4b6b-8451-4713172f35b0\") " pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-gn8v4"
Oct 30 21:49:44.943596 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:44.943484    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f587fb39-646c-4b6b-8451-4713172f35b0-openshift-service-ca\") pod \"maas-api-6d669c7fcd-gn8v4\" (UID: \"f587fb39-646c-4b6b-8451-4713172f35b0\") " pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-gn8v4"
Oct 30 21:49:45.003258 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:45.003212    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Oct 30 21:49:45.003258 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:45.003237    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-pjcd6\""
Oct 30 21:49:45.003524 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:45.003295    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Oct 30 21:49:45.004637 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:45.004613    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6d669c7fcd-gn8v4"]
Oct 30 21:49:45.043871 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:45.043833    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f587fb39-646c-4b6b-8451-4713172f35b0-maas-api-tls\") pod \"maas-api-6d669c7fcd-gn8v4\" (UID: \"f587fb39-646c-4b6b-8451-4713172f35b0\") " pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-gn8v4"
Oct 30 21:49:45.043871 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:45.043870    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f587fb39-646c-4b6b-8451-4713172f35b0-openshift-service-ca\") pod \"maas-api-6d669c7fcd-gn8v4\" (UID: \"f587fb39-646c-4b6b-8451-4713172f35b0\") " pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-gn8v4"
Oct 30 21:49:45.044156 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:45.043979    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f587fb39-646c-4b6b-8451-4713172f35b0-metrics-certs\") pod \"maas-api-6d669c7fcd-gn8v4\" (UID: \"f587fb39-646c-4b6b-8451-4713172f35b0\") " pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-gn8v4"
Oct 30 21:49:45.044156 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:45.044015    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n8k58\" (UniqueName: \"kubernetes.io/projected/f587fb39-646c-4b6b-8451-4713172f35b0-kube-api-access-n8k58\") pod \"maas-api-6d669c7fcd-gn8v4\" (UID: \"f587fb39-646c-4b6b-8451-4713172f35b0\") " pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-gn8v4"
Oct 30 21:49:45.044727 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:45.044649    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f587fb39-646c-4b6b-8451-4713172f35b0-openshift-service-ca\") pod \"maas-api-6d669c7fcd-gn8v4\" (UID: \"f587fb39-646c-4b6b-8451-4713172f35b0\") " pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-gn8v4"
Oct 30 21:49:45.046493 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:45.046471    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f587fb39-646c-4b6b-8451-4713172f35b0-maas-api-tls\") pod \"maas-api-6d669c7fcd-gn8v4\" (UID: \"f587fb39-646c-4b6b-8451-4713172f35b0\") " pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-gn8v4"
Oct 30 21:49:45.046559 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:45.046505    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f587fb39-646c-4b6b-8451-4713172f35b0-metrics-certs\") pod \"maas-api-6d669c7fcd-gn8v4\" (UID: \"f587fb39-646c-4b6b-8451-4713172f35b0\") " pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-gn8v4"
Oct 30 21:49:45.110738 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:45.110706    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n8k58\" (UniqueName: \"kubernetes.io/projected/f587fb39-646c-4b6b-8451-4713172f35b0-kube-api-access-n8k58\") pod \"maas-api-6d669c7fcd-gn8v4\" (UID: \"f587fb39-646c-4b6b-8451-4713172f35b0\") " pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-gn8v4"
Oct 30 21:49:45.211859 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:45.211776    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-gn8v4"
Oct 30 21:49:45.584908 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:45.584829    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6d669c7fcd-gn8v4"]
Oct 30 21:49:45.587576 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:49:45.587544    4473 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf587fb39_646c_4b6b_8451_4713172f35b0.slice/crio-0eaa9cf1bc94a1d6e3aab4658bcb8a8d644d2ebbf38865c253f74e775766dbbe WatchSource:0}: Error finding container 0eaa9cf1bc94a1d6e3aab4658bcb8a8d644d2ebbf38865c253f74e775766dbbe: Status 404 returned error can't find the container with id 0eaa9cf1bc94a1d6e3aab4658bcb8a8d644d2ebbf38865c253f74e775766dbbe
Oct 30 21:49:46.211855 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:46.211819    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-gn8v4" event={"ID":"f587fb39-646c-4b6b-8451-4713172f35b0","Type":"ContainerStarted","Data":"0eaa9cf1bc94a1d6e3aab4658bcb8a8d644d2ebbf38865c253f74e775766dbbe"}
Oct 30 21:49:46.386559 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:46.386523    4473 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-lxkmp"]
Oct 30 21:49:46.389843 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:46.389824    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-lxkmp"
Oct 30 21:49:46.455637 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:46.455601    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-trzvr\" (UniqueName: \"kubernetes.io/projected/8b90548f-3da8-42df-99d0-d5eb415a4af3-kube-api-access-trzvr\") pod \"istiod-openshift-gateway-7ddd9cfdc8-lxkmp\" (UID: \"8b90548f-3da8-42df-99d0-d5eb415a4af3\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-lxkmp"
Oct 30 21:49:46.455820 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:46.455678    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8b90548f-3da8-42df-99d0-d5eb415a4af3-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-lxkmp\" (UID: \"8b90548f-3da8-42df-99d0-d5eb415a4af3\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-lxkmp"
Oct 30 21:49:46.455820 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:46.455759    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/8b90548f-3da8-42df-99d0-d5eb415a4af3-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-lxkmp\" (UID: \"8b90548f-3da8-42df-99d0-d5eb415a4af3\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-lxkmp"
Oct 30 21:49:46.455820 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:46.455787    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8b90548f-3da8-42df-99d0-d5eb415a4af3-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-lxkmp\" (UID: \"8b90548f-3da8-42df-99d0-d5eb415a4af3\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-lxkmp"
Oct 30 21:49:46.455820 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:46.455805    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/8b90548f-3da8-42df-99d0-d5eb415a4af3-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-lxkmp\" (UID: \"8b90548f-3da8-42df-99d0-d5eb415a4af3\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-lxkmp"
Oct 30 21:49:46.455977 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:46.455959    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/8b90548f-3da8-42df-99d0-d5eb415a4af3-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-lxkmp\" (UID: \"8b90548f-3da8-42df-99d0-d5eb415a4af3\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-lxkmp"
Oct 30 21:49:46.456014 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:46.455987    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/8b90548f-3da8-42df-99d0-d5eb415a4af3-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-lxkmp\" (UID: \"8b90548f-3da8-42df-99d0-d5eb415a4af3\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-lxkmp"
Oct 30 21:49:46.491558 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:46.491461    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-lxkmp"]
Oct 30 21:49:46.557221 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:46.557166    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8b90548f-3da8-42df-99d0-d5eb415a4af3-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-lxkmp\" (UID: \"8b90548f-3da8-42df-99d0-d5eb415a4af3\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-lxkmp"
Oct 30 21:49:46.557221 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:46.557218    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/8b90548f-3da8-42df-99d0-d5eb415a4af3-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-lxkmp\" (UID: \"8b90548f-3da8-42df-99d0-d5eb415a4af3\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-lxkmp"
Oct 30 21:49:46.557476 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:46.557406    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8b90548f-3da8-42df-99d0-d5eb415a4af3-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-lxkmp\" (UID: \"8b90548f-3da8-42df-99d0-d5eb415a4af3\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-lxkmp"
Oct 30 21:49:46.557476 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:46.557445    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/8b90548f-3da8-42df-99d0-d5eb415a4af3-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-lxkmp\" (UID: \"8b90548f-3da8-42df-99d0-d5eb415a4af3\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-lxkmp"
Oct 30 21:49:46.557616 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:46.557513    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/8b90548f-3da8-42df-99d0-d5eb415a4af3-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-lxkmp\" (UID: \"8b90548f-3da8-42df-99d0-d5eb415a4af3\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-lxkmp"
Oct 30 21:49:46.557616 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:46.557540    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/8b90548f-3da8-42df-99d0-d5eb415a4af3-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-lxkmp\" (UID: \"8b90548f-3da8-42df-99d0-d5eb415a4af3\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-lxkmp"
Oct 30 21:49:46.557810 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:46.557770    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-trzvr\" (UniqueName: \"kubernetes.io/projected/8b90548f-3da8-42df-99d0-d5eb415a4af3-kube-api-access-trzvr\") pod \"istiod-openshift-gateway-7ddd9cfdc8-lxkmp\" (UID: \"8b90548f-3da8-42df-99d0-d5eb415a4af3\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-lxkmp"
Oct 30 21:49:46.557970 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:46.557946    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/8b90548f-3da8-42df-99d0-d5eb415a4af3-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-lxkmp\" (UID: \"8b90548f-3da8-42df-99d0-d5eb415a4af3\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-lxkmp"
Oct 30 21:49:46.560500 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:46.560479    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8b90548f-3da8-42df-99d0-d5eb415a4af3-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-lxkmp\" (UID: \"8b90548f-3da8-42df-99d0-d5eb415a4af3\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-lxkmp"
Oct 30 21:49:46.560774 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:46.560748    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/8b90548f-3da8-42df-99d0-d5eb415a4af3-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-lxkmp\" (UID: \"8b90548f-3da8-42df-99d0-d5eb415a4af3\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-lxkmp"
Oct 30 21:49:46.560854 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:46.560827    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/8b90548f-3da8-42df-99d0-d5eb415a4af3-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-lxkmp\" (UID: \"8b90548f-3da8-42df-99d0-d5eb415a4af3\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-lxkmp"
Oct 30 21:49:46.560909 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:46.560867    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/8b90548f-3da8-42df-99d0-d5eb415a4af3-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-lxkmp\" (UID: \"8b90548f-3da8-42df-99d0-d5eb415a4af3\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-lxkmp"
Oct 30 21:49:46.616280 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:46.616239    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8b90548f-3da8-42df-99d0-d5eb415a4af3-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-lxkmp\" (UID: \"8b90548f-3da8-42df-99d0-d5eb415a4af3\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-lxkmp"
Oct 30 21:49:46.616512 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:46.616487    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-trzvr\" (UniqueName: \"kubernetes.io/projected/8b90548f-3da8-42df-99d0-d5eb415a4af3-kube-api-access-trzvr\") pod \"istiod-openshift-gateway-7ddd9cfdc8-lxkmp\" (UID: \"8b90548f-3da8-42df-99d0-d5eb415a4af3\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-lxkmp"
Oct 30 21:49:46.702184 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:46.702143    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-lxkmp"
Oct 30 21:49:47.175744 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:47.175709    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-lxkmp"]
Oct 30 21:49:47.184091 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:49:47.184053    4473 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b90548f_3da8_42df_99d0_d5eb415a4af3.slice/crio-cb8eaf86028a3f2eabb8cbb44cbebcc0a6ccd8e5836b79ba3f23e37f20054c9e WatchSource:0}: Error finding container cb8eaf86028a3f2eabb8cbb44cbebcc0a6ccd8e5836b79ba3f23e37f20054c9e: Status 404 returned error can't find the container with id cb8eaf86028a3f2eabb8cbb44cbebcc0a6ccd8e5836b79ba3f23e37f20054c9e
Oct 30 21:49:47.185836 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:47.185793    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:49:47.185953 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:47.185868    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:49:47.218544 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:47.218502    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-lxkmp" event={"ID":"8b90548f-3da8-42df-99d0-d5eb415a4af3","Type":"ContainerStarted","Data":"cb8eaf86028a3f2eabb8cbb44cbebcc0a6ccd8e5836b79ba3f23e37f20054c9e"}
Oct 30 21:49:48.225547 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:48.225516    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-lxkmp" event={"ID":"8b90548f-3da8-42df-99d0-d5eb415a4af3","Type":"ContainerStarted","Data":"153757e7bbcb55858fe522c63c89b4d27454da55a5d79eafc8df23c63b75eff3"}
Oct 30 21:49:48.226066 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:48.225700    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-lxkmp"
Oct 30 21:49:48.227294 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:48.227269    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-gn8v4" event={"ID":"f587fb39-646c-4b6b-8451-4713172f35b0","Type":"ContainerStarted","Data":"267adefae37383881736885b9247a42587058927bb323952b3601475157ac31b"}
Oct 30 21:49:48.227403 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:48.227377    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-gn8v4"
Oct 30 21:49:48.227620 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:48.227593    4473 patch_prober.go:29] interesting pod/istiod-openshift-gateway-7ddd9cfdc8-lxkmp container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 21:49:48.227744 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:48.227654    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-lxkmp" podUID="8b90548f-3da8-42df-99d0-d5eb415a4af3" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 21:49:48.455394 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:48.455339    4473 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-lxkmp" podStartSLOduration=2.4553236099999998 podStartE2EDuration="2.45532361s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:49:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:49:48.454399755 +0000 UTC m=+434.227768241" watchObservedRunningTime="2026-10-30 21:49:48.45532361 +0000 UTC m=+434.228692094"
Oct 30 21:49:48.586037 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:48.585888    4473 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-gn8v4" podStartSLOduration=2.666980426 podStartE2EDuration="4.585874294s" totalImagesPullingTime="1.918893868s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:49:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:49:48.585469781 +0000 UTC m=+434.358838270" watchObservedRunningTime="2026-10-30 21:49:48.585874294 +0000 UTC m=+434.359242780"
Oct 30 21:49:49.232062 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:49.232030    4473 patch_prober.go:29] interesting pod/istiod-openshift-gateway-7ddd9cfdc8-lxkmp container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 21:49:49.232511 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:49.232082    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-lxkmp" podUID="8b90548f-3da8-42df-99d0-d5eb415a4af3" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 21:49:50.324378 ip-10-0-140-18 kubenswrapper[4473]: E1030 21:49:50.324342    4473 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fncvg" containerName="extractor"
Oct 30 21:49:52.232603 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:52.232571    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-lxkmp"
Oct 30 21:49:52.456898 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:52.456867    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-pg6b6"]
Oct 30 21:49:52.457094 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:52.457078    4473 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-pg6b6" podUID="36fd53ff-3e4a-4581-9c87-cd1ccf42e02e" containerName="discovery" containerID="cri-o://e8670e41a0c5435b30e6a6496d19ab018f2c03793bd03025842f7a945d8af40a" gracePeriod=30
Oct 30 21:49:52.713136 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:52.713112    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-pg6b6"
Oct 30 21:49:52.816983 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:52.816878    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/36fd53ff-3e4a-4581-9c87-cd1ccf42e02e-istio-token\" (UniqueName: \"kubernetes.io/projected/36fd53ff-3e4a-4581-9c87-cd1ccf42e02e-istio-token\") pod \"36fd53ff-3e4a-4581-9c87-cd1ccf42e02e\" (UID: \"36fd53ff-3e4a-4581-9c87-cd1ccf42e02e\") "
Oct 30 21:49:52.816983 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:52.816945    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/36fd53ff-3e4a-4581-9c87-cd1ccf42e02e-istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/36fd53ff-3e4a-4581-9c87-cd1ccf42e02e-istio-csr-dns-cert\") pod \"36fd53ff-3e4a-4581-9c87-cd1ccf42e02e\" (UID: \"36fd53ff-3e4a-4581-9c87-cd1ccf42e02e\") "
Oct 30 21:49:52.817189 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:52.816993    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/36fd53ff-3e4a-4581-9c87-cd1ccf42e02e-kube-api-access-fpxq4\" (UniqueName: \"kubernetes.io/projected/36fd53ff-3e4a-4581-9c87-cd1ccf42e02e-kube-api-access-fpxq4\") pod \"36fd53ff-3e4a-4581-9c87-cd1ccf42e02e\" (UID: \"36fd53ff-3e4a-4581-9c87-cd1ccf42e02e\") "
Oct 30 21:49:52.817189 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:52.817033    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/36fd53ff-3e4a-4581-9c87-cd1ccf42e02e-istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/36fd53ff-3e4a-4581-9c87-cd1ccf42e02e-istio-kubeconfig\") pod \"36fd53ff-3e4a-4581-9c87-cd1ccf42e02e\" (UID: \"36fd53ff-3e4a-4581-9c87-cd1ccf42e02e\") "
Oct 30 21:49:52.817189 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:52.817066    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/36fd53ff-3e4a-4581-9c87-cd1ccf42e02e-istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/36fd53ff-3e4a-4581-9c87-cd1ccf42e02e-istio-csr-ca-configmap\") pod \"36fd53ff-3e4a-4581-9c87-cd1ccf42e02e\" (UID: \"36fd53ff-3e4a-4581-9c87-cd1ccf42e02e\") "
Oct 30 21:49:52.817189 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:52.817089    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/36fd53ff-3e4a-4581-9c87-cd1ccf42e02e-cacerts\" (UniqueName: \"kubernetes.io/secret/36fd53ff-3e4a-4581-9c87-cd1ccf42e02e-cacerts\") pod \"36fd53ff-3e4a-4581-9c87-cd1ccf42e02e\" (UID: \"36fd53ff-3e4a-4581-9c87-cd1ccf42e02e\") "
Oct 30 21:49:52.817189 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:52.817155    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/36fd53ff-3e4a-4581-9c87-cd1ccf42e02e-local-certs\" (UniqueName: \"kubernetes.io/empty-dir/36fd53ff-3e4a-4581-9c87-cd1ccf42e02e-local-certs\") pod \"36fd53ff-3e4a-4581-9c87-cd1ccf42e02e\" (UID: \"36fd53ff-3e4a-4581-9c87-cd1ccf42e02e\") "
Oct 30 21:49:52.817567 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:52.817517    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/36fd53ff-3e4a-4581-9c87-cd1ccf42e02e-istio-csr-ca-configmap" pod "36fd53ff-3e4a-4581-9c87-cd1ccf42e02e" (UID: "36fd53ff-3e4a-4581-9c87-cd1ccf42e02e"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:49:52.820213 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:52.820181    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/36fd53ff-3e4a-4581-9c87-cd1ccf42e02e-kube-api-access-fpxq4" pod "36fd53ff-3e4a-4581-9c87-cd1ccf42e02e" (UID: "36fd53ff-3e4a-4581-9c87-cd1ccf42e02e"). InnerVolumeSpecName "kube-api-access-fpxq4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:49:52.820213 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:52.820188    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/36fd53ff-3e4a-4581-9c87-cd1ccf42e02e-istio-csr-dns-cert" pod "36fd53ff-3e4a-4581-9c87-cd1ccf42e02e" (UID: "36fd53ff-3e4a-4581-9c87-cd1ccf42e02e"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:49:52.820465 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:52.820296    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/36fd53ff-3e4a-4581-9c87-cd1ccf42e02e-cacerts" pod "36fd53ff-3e4a-4581-9c87-cd1ccf42e02e" (UID: "36fd53ff-3e4a-4581-9c87-cd1ccf42e02e"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:49:52.820465 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:52.820317    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/36fd53ff-3e4a-4581-9c87-cd1ccf42e02e-istio-token" pod "36fd53ff-3e4a-4581-9c87-cd1ccf42e02e" (UID: "36fd53ff-3e4a-4581-9c87-cd1ccf42e02e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:49:52.820667 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:52.820642    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/36fd53ff-3e4a-4581-9c87-cd1ccf42e02e-istio-kubeconfig" pod "36fd53ff-3e4a-4581-9c87-cd1ccf42e02e" (UID: "36fd53ff-3e4a-4581-9c87-cd1ccf42e02e"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:49:52.820667 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:52.820649    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/36fd53ff-3e4a-4581-9c87-cd1ccf42e02e-local-certs" pod "36fd53ff-3e4a-4581-9c87-cd1ccf42e02e" (UID: "36fd53ff-3e4a-4581-9c87-cd1ccf42e02e"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:49:52.915622 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:52.915588    4473 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-769875db56-wr8rv"]
Oct 30 21:49:52.916013 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:52.916000    4473 cpu_manager.go:414] "containerMap: removing container" podUID="36fd53ff-3e4a-4581-9c87-cd1ccf42e02e" containerName="discovery"
Oct 30 21:49:52.916054 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:52.916015    4473 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="36fd53ff-3e4a-4581-9c87-cd1ccf42e02e" containerName="discovery"
Oct 30 21:49:52.916103 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:52.916083    4473 memory_manager.go:365] "RemoveStaleState removing state" podUID="36fd53ff-3e4a-4581-9c87-cd1ccf42e02e" containerName="discovery"
Oct 30 21:49:52.918520 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:52.918487    4473 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/36fd53ff-3e4a-4581-9c87-cd1ccf42e02e-istio-token\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:49:52.918520 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:52.918515    4473 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/36fd53ff-3e4a-4581-9c87-cd1ccf42e02e-istio-csr-dns-cert\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:49:52.918520 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:52.918525    4473 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fpxq4\" (UniqueName: \"kubernetes.io/projected/36fd53ff-3e4a-4581-9c87-cd1ccf42e02e-kube-api-access-fpxq4\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:49:52.918711 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:52.918535    4473 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/36fd53ff-3e4a-4581-9c87-cd1ccf42e02e-istio-kubeconfig\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:49:52.918711 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:52.918545    4473 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/36fd53ff-3e4a-4581-9c87-cd1ccf42e02e-istio-csr-ca-configmap\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:49:52.918711 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:52.918554    4473 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/36fd53ff-3e4a-4581-9c87-cd1ccf42e02e-cacerts\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:49:52.918711 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:52.918563    4473 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/36fd53ff-3e4a-4581-9c87-cd1ccf42e02e-local-certs\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:49:52.919418 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:52.919398    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-769875db56-wr8rv"
Oct 30 21:49:52.974245 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:52.974211    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-metrics-tls\""
Oct 30 21:49:52.974245 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:52.974254    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-qjzs6\""
Oct 30 21:49:52.974485 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:52.974265    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Oct 30 21:49:52.975907 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:52.975878    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-769875db56-wr8rv"]
Oct 30 21:49:53.019543 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:53.019503    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d54f8662-fb93-472e-a930-8d07c7550edb-metrics-certs\") pod \"odh-model-controller-769875db56-wr8rv\" (UID: \"d54f8662-fb93-472e-a930-8d07c7550edb\") " pod="opendatahub/odh-model-controller-769875db56-wr8rv"
Oct 30 21:49:53.019724 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:53.019659    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d54f8662-fb93-472e-a930-8d07c7550edb-cert\") pod \"odh-model-controller-769875db56-wr8rv\" (UID: \"d54f8662-fb93-472e-a930-8d07c7550edb\") " pod="opendatahub/odh-model-controller-769875db56-wr8rv"
Oct 30 21:49:53.019783 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:53.019752    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-99d9n\" (UniqueName: \"kubernetes.io/projected/d54f8662-fb93-472e-a930-8d07c7550edb-kube-api-access-99d9n\") pod \"odh-model-controller-769875db56-wr8rv\" (UID: \"d54f8662-fb93-472e-a930-8d07c7550edb\") " pod="opendatahub/odh-model-controller-769875db56-wr8rv"
Oct 30 21:49:53.121093 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:53.121059    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-99d9n\" (UniqueName: \"kubernetes.io/projected/d54f8662-fb93-472e-a930-8d07c7550edb-kube-api-access-99d9n\") pod \"odh-model-controller-769875db56-wr8rv\" (UID: \"d54f8662-fb93-472e-a930-8d07c7550edb\") " pod="opendatahub/odh-model-controller-769875db56-wr8rv"
Oct 30 21:49:53.121282 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:53.121107    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d54f8662-fb93-472e-a930-8d07c7550edb-metrics-certs\") pod \"odh-model-controller-769875db56-wr8rv\" (UID: \"d54f8662-fb93-472e-a930-8d07c7550edb\") " pod="opendatahub/odh-model-controller-769875db56-wr8rv"
Oct 30 21:49:53.121282 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:53.121168    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d54f8662-fb93-472e-a930-8d07c7550edb-cert\") pod \"odh-model-controller-769875db56-wr8rv\" (UID: \"d54f8662-fb93-472e-a930-8d07c7550edb\") " pod="opendatahub/odh-model-controller-769875db56-wr8rv"
Oct 30 21:49:53.123763 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:53.123737    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d54f8662-fb93-472e-a930-8d07c7550edb-metrics-certs\") pod \"odh-model-controller-769875db56-wr8rv\" (UID: \"d54f8662-fb93-472e-a930-8d07c7550edb\") " pod="opendatahub/odh-model-controller-769875db56-wr8rv"
Oct 30 21:49:53.123878 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:53.123797    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d54f8662-fb93-472e-a930-8d07c7550edb-cert\") pod \"odh-model-controller-769875db56-wr8rv\" (UID: \"d54f8662-fb93-472e-a930-8d07c7550edb\") " pod="opendatahub/odh-model-controller-769875db56-wr8rv"
Oct 30 21:49:53.179598 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:53.179565    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-99d9n\" (UniqueName: \"kubernetes.io/projected/d54f8662-fb93-472e-a930-8d07c7550edb-kube-api-access-99d9n\") pod \"odh-model-controller-769875db56-wr8rv\" (UID: \"d54f8662-fb93-472e-a930-8d07c7550edb\") " pod="opendatahub/odh-model-controller-769875db56-wr8rv"
Oct 30 21:49:53.232408 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:53.232372    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-769875db56-wr8rv"
Oct 30 21:49:53.250453 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:53.250425    4473 generic.go:356] "Generic (PLEG): container finished" podID="36fd53ff-3e4a-4581-9c87-cd1ccf42e02e" containerID="e8670e41a0c5435b30e6a6496d19ab018f2c03793bd03025842f7a945d8af40a" exitCode=0
Oct 30 21:49:53.250893 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:53.250479    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-pg6b6"
Oct 30 21:49:53.250893 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:53.250501    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-pg6b6" event={"ID":"36fd53ff-3e4a-4581-9c87-cd1ccf42e02e","Type":"ContainerDied","Data":"e8670e41a0c5435b30e6a6496d19ab018f2c03793bd03025842f7a945d8af40a"}
Oct 30 21:49:53.250893 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:53.250556    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-pg6b6" event={"ID":"36fd53ff-3e4a-4581-9c87-cd1ccf42e02e","Type":"ContainerDied","Data":"b3395d2761513345d9f8af41ea8634da7fc88db86b0ffed2df792d41d42d4bab"}
Oct 30 21:49:53.250893 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:53.250579    4473 scope.go:122] "RemoveContainer" containerID="e8670e41a0c5435b30e6a6496d19ab018f2c03793bd03025842f7a945d8af40a"
Oct 30 21:49:53.261862 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:53.261841    4473 scope.go:122] "RemoveContainer" containerID="e8670e41a0c5435b30e6a6496d19ab018f2c03793bd03025842f7a945d8af40a"
Oct 30 21:49:53.262296 ip-10-0-140-18 kubenswrapper[4473]: E1030 21:49:53.262261    4473 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e8670e41a0c5435b30e6a6496d19ab018f2c03793bd03025842f7a945d8af40a\": container with ID starting with e8670e41a0c5435b30e6a6496d19ab018f2c03793bd03025842f7a945d8af40a not found: ID does not exist" containerID="e8670e41a0c5435b30e6a6496d19ab018f2c03793bd03025842f7a945d8af40a"
Oct 30 21:49:53.262412 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:53.262298    4473 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e8670e41a0c5435b30e6a6496d19ab018f2c03793bd03025842f7a945d8af40a"} err="failed to get container status \"e8670e41a0c5435b30e6a6496d19ab018f2c03793bd03025842f7a945d8af40a\": rpc error: code = NotFound desc = could not find container \"e8670e41a0c5435b30e6a6496d19ab018f2c03793bd03025842f7a945d8af40a\": container with ID starting with e8670e41a0c5435b30e6a6496d19ab018f2c03793bd03025842f7a945d8af40a not found: ID does not exist"
Oct 30 21:49:53.417873 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:53.417782    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-pg6b6"]
Oct 30 21:49:53.420767 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:53.420738    4473 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-pg6b6"]
Oct 30 21:49:53.604201 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:53.604173    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-769875db56-wr8rv"]
Oct 30 21:49:53.606888 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:49:53.606860    4473 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd54f8662_fb93_472e_a930_8d07c7550edb.slice/crio-78b9da486778e6c8fe18db9c34dca72d4390b3828e95fc431e5bb74d5d25167c WatchSource:0}: Error finding container 78b9da486778e6c8fe18db9c34dca72d4390b3828e95fc431e5bb74d5d25167c: Status 404 returned error can't find the container with id 78b9da486778e6c8fe18db9c34dca72d4390b3828e95fc431e5bb74d5d25167c
Oct 30 21:49:54.236755 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:54.236722    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-gn8v4"
Oct 30 21:49:54.256031 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:54.255996    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-769875db56-wr8rv" event={"ID":"d54f8662-fb93-472e-a930-8d07c7550edb","Type":"ContainerStarted","Data":"78b9da486778e6c8fe18db9c34dca72d4390b3828e95fc431e5bb74d5d25167c"}
Oct 30 21:49:54.359675 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:54.359639    4473 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="36fd53ff-3e4a-4581-9c87-cd1ccf42e02e" path="/var/lib/kubelet/pods/36fd53ff-3e4a-4581-9c87-cd1ccf42e02e/volumes"
Oct 30 21:49:55.211951 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:55.211896    4473 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-79787b59b7-d8gzw"]
Oct 30 21:49:55.215893 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:55.215863    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-79787b59b7-d8gzw"
Oct 30 21:49:55.322259 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:55.322225    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-79787b59b7-d8gzw"]
Oct 30 21:49:55.341764 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:55.341731    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c3ecd156-ac60-4605-86bb-e531fdcd1ce0-openshift-service-ca\") pod \"maas-api-79787b59b7-d8gzw\" (UID: \"c3ecd156-ac60-4605-86bb-e531fdcd1ce0\") " pod="odh-ai-gateway-infra/maas-api-79787b59b7-d8gzw"
Oct 30 21:49:55.341986 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:55.341779    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c3ecd156-ac60-4605-86bb-e531fdcd1ce0-maas-api-tls\") pod \"maas-api-79787b59b7-d8gzw\" (UID: \"c3ecd156-ac60-4605-86bb-e531fdcd1ce0\") " pod="odh-ai-gateway-infra/maas-api-79787b59b7-d8gzw"
Oct 30 21:49:55.341986 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:55.341841    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c3ecd156-ac60-4605-86bb-e531fdcd1ce0-metrics-certs\") pod \"maas-api-79787b59b7-d8gzw\" (UID: \"c3ecd156-ac60-4605-86bb-e531fdcd1ce0\") " pod="odh-ai-gateway-infra/maas-api-79787b59b7-d8gzw"
Oct 30 21:49:55.341986 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:55.341883    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-glb95\" (UniqueName: \"kubernetes.io/projected/c3ecd156-ac60-4605-86bb-e531fdcd1ce0-kube-api-access-glb95\") pod \"maas-api-79787b59b7-d8gzw\" (UID: \"c3ecd156-ac60-4605-86bb-e531fdcd1ce0\") " pod="odh-ai-gateway-infra/maas-api-79787b59b7-d8gzw"
Oct 30 21:49:55.443430 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:55.443393    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c3ecd156-ac60-4605-86bb-e531fdcd1ce0-openshift-service-ca\") pod \"maas-api-79787b59b7-d8gzw\" (UID: \"c3ecd156-ac60-4605-86bb-e531fdcd1ce0\") " pod="odh-ai-gateway-infra/maas-api-79787b59b7-d8gzw"
Oct 30 21:49:55.443588 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:55.443450    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c3ecd156-ac60-4605-86bb-e531fdcd1ce0-maas-api-tls\") pod \"maas-api-79787b59b7-d8gzw\" (UID: \"c3ecd156-ac60-4605-86bb-e531fdcd1ce0\") " pod="odh-ai-gateway-infra/maas-api-79787b59b7-d8gzw"
Oct 30 21:49:55.443588 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:55.443470    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c3ecd156-ac60-4605-86bb-e531fdcd1ce0-metrics-certs\") pod \"maas-api-79787b59b7-d8gzw\" (UID: \"c3ecd156-ac60-4605-86bb-e531fdcd1ce0\") " pod="odh-ai-gateway-infra/maas-api-79787b59b7-d8gzw"
Oct 30 21:49:55.443588 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:55.443498    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-glb95\" (UniqueName: \"kubernetes.io/projected/c3ecd156-ac60-4605-86bb-e531fdcd1ce0-kube-api-access-glb95\") pod \"maas-api-79787b59b7-d8gzw\" (UID: \"c3ecd156-ac60-4605-86bb-e531fdcd1ce0\") " pod="odh-ai-gateway-infra/maas-api-79787b59b7-d8gzw"
Oct 30 21:49:55.444211 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:55.444187    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c3ecd156-ac60-4605-86bb-e531fdcd1ce0-openshift-service-ca\") pod \"maas-api-79787b59b7-d8gzw\" (UID: \"c3ecd156-ac60-4605-86bb-e531fdcd1ce0\") " pod="odh-ai-gateway-infra/maas-api-79787b59b7-d8gzw"
Oct 30 21:49:55.446417 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:55.446356    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c3ecd156-ac60-4605-86bb-e531fdcd1ce0-maas-api-tls\") pod \"maas-api-79787b59b7-d8gzw\" (UID: \"c3ecd156-ac60-4605-86bb-e531fdcd1ce0\") " pod="odh-ai-gateway-infra/maas-api-79787b59b7-d8gzw"
Oct 30 21:49:55.446417 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:55.446394    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c3ecd156-ac60-4605-86bb-e531fdcd1ce0-metrics-certs\") pod \"maas-api-79787b59b7-d8gzw\" (UID: \"c3ecd156-ac60-4605-86bb-e531fdcd1ce0\") " pod="odh-ai-gateway-infra/maas-api-79787b59b7-d8gzw"
Oct 30 21:49:55.502308 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:55.502218    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-glb95\" (UniqueName: \"kubernetes.io/projected/c3ecd156-ac60-4605-86bb-e531fdcd1ce0-kube-api-access-glb95\") pod \"maas-api-79787b59b7-d8gzw\" (UID: \"c3ecd156-ac60-4605-86bb-e531fdcd1ce0\") " pod="odh-ai-gateway-infra/maas-api-79787b59b7-d8gzw"
Oct 30 21:49:55.532712 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:55.532675    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-79787b59b7-d8gzw"
Oct 30 21:49:55.924966 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:55.924934    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-79787b59b7-d8gzw"]
Oct 30 21:49:55.927808 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:49:55.927775    4473 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3ecd156_ac60_4605_86bb_e531fdcd1ce0.slice/crio-483d6d04c25e4b89a2619f49a69009cc4f14e20cf238767fd9e31e27b8cd85dd WatchSource:0}: Error finding container 483d6d04c25e4b89a2619f49a69009cc4f14e20cf238767fd9e31e27b8cd85dd: Status 404 returned error can't find the container with id 483d6d04c25e4b89a2619f49a69009cc4f14e20cf238767fd9e31e27b8cd85dd
Oct 30 21:49:56.269693 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:56.269604    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-79787b59b7-d8gzw" event={"ID":"c3ecd156-ac60-4605-86bb-e531fdcd1ce0","Type":"ContainerStarted","Data":"483d6d04c25e4b89a2619f49a69009cc4f14e20cf238767fd9e31e27b8cd85dd"}
Oct 30 21:49:57.276577 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:57.276537    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-769875db56-wr8rv" event={"ID":"d54f8662-fb93-472e-a930-8d07c7550edb","Type":"ContainerStarted","Data":"07555d97075393e4e0c99ffe0712f4e7bd58fa0227cfe8c9dc39a4ee9ae8af93"}
Oct 30 21:49:57.277130 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:57.276658    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-769875db56-wr8rv"
Oct 30 21:49:57.278505 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:57.278470    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-79787b59b7-d8gzw" event={"ID":"c3ecd156-ac60-4605-86bb-e531fdcd1ce0","Type":"ContainerStarted","Data":"e6a569add08db5105c87008b0660d7985da223d3779609af911b0c7c1c650310"}
Oct 30 21:49:57.278659 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:57.278596    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-79787b59b7-d8gzw"
Oct 30 21:49:57.399890 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:57.399824    4473 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/odh-model-controller-769875db56-wr8rv" podStartSLOduration=1.8499934759999999 podStartE2EDuration="5.399807038s" totalImagesPullingTime="3.549813562s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:49:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:49:57.398471678 +0000 UTC m=+443.171840166" watchObservedRunningTime="2026-10-30 21:49:57.399807038 +0000 UTC m=+443.173175556"
Oct 30 21:49:57.520648 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:49:57.520539    4473 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-79787b59b7-d8gzw" podStartSLOduration=1.291919791 podStartE2EDuration="2.520523156s" totalImagesPullingTime="1.228603365s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:49:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:49:57.519444289 +0000 UTC m=+443.292812779" watchObservedRunningTime="2026-10-30 21:49:57.520523156 +0000 UTC m=+443.293891643"
Oct 30 21:50:03.288252 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:50:03.288225    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-79787b59b7-d8gzw"
Oct 30 21:50:03.460129 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:50:03.460095    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-6d669c7fcd-gn8v4"]
Oct 30 21:50:03.460390 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:50:03.460363    4473 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-gn8v4" podUID="f587fb39-646c-4b6b-8451-4713172f35b0" containerName="maas-api" containerID="cri-o://267adefae37383881736885b9247a42587058927bb323952b3601475157ac31b" gracePeriod=30
Oct 30 21:50:03.696532 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:50:03.696507    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-gn8v4"
Oct 30 21:50:03.820942 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:50:03.819691    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f587fb39-646c-4b6b-8451-4713172f35b0-metrics-certs\" (UniqueName: \"kubernetes.io/secret/f587fb39-646c-4b6b-8451-4713172f35b0-metrics-certs\") pod \"f587fb39-646c-4b6b-8451-4713172f35b0\" (UID: \"f587fb39-646c-4b6b-8451-4713172f35b0\") "
Oct 30 21:50:03.820942 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:50:03.819829    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f587fb39-646c-4b6b-8451-4713172f35b0-kube-api-access-n8k58\" (UniqueName: \"kubernetes.io/projected/f587fb39-646c-4b6b-8451-4713172f35b0-kube-api-access-n8k58\") pod \"f587fb39-646c-4b6b-8451-4713172f35b0\" (UID: \"f587fb39-646c-4b6b-8451-4713172f35b0\") "
Oct 30 21:50:03.820942 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:50:03.819868    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f587fb39-646c-4b6b-8451-4713172f35b0-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f587fb39-646c-4b6b-8451-4713172f35b0-openshift-service-ca\") pod \"f587fb39-646c-4b6b-8451-4713172f35b0\" (UID: \"f587fb39-646c-4b6b-8451-4713172f35b0\") "
Oct 30 21:50:03.820942 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:50:03.819939    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f587fb39-646c-4b6b-8451-4713172f35b0-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f587fb39-646c-4b6b-8451-4713172f35b0-maas-api-tls\") pod \"f587fb39-646c-4b6b-8451-4713172f35b0\" (UID: \"f587fb39-646c-4b6b-8451-4713172f35b0\") "
Oct 30 21:50:03.821858 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:50:03.821828    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f587fb39-646c-4b6b-8451-4713172f35b0-openshift-service-ca" pod "f587fb39-646c-4b6b-8451-4713172f35b0" (UID: "f587fb39-646c-4b6b-8451-4713172f35b0"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:50:03.824350 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:50:03.824284    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f587fb39-646c-4b6b-8451-4713172f35b0-kube-api-access-n8k58" pod "f587fb39-646c-4b6b-8451-4713172f35b0" (UID: "f587fb39-646c-4b6b-8451-4713172f35b0"). InnerVolumeSpecName "kube-api-access-n8k58". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:50:03.829057 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:50:03.829003    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f587fb39-646c-4b6b-8451-4713172f35b0-metrics-certs" pod "f587fb39-646c-4b6b-8451-4713172f35b0" (UID: "f587fb39-646c-4b6b-8451-4713172f35b0"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:50:03.829057 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:50:03.829036    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f587fb39-646c-4b6b-8451-4713172f35b0-maas-api-tls" pod "f587fb39-646c-4b6b-8451-4713172f35b0" (UID: "f587fb39-646c-4b6b-8451-4713172f35b0"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:50:03.921467 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:50:03.921429    4473 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n8k58\" (UniqueName: \"kubernetes.io/projected/f587fb39-646c-4b6b-8451-4713172f35b0-kube-api-access-n8k58\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:50:03.921467 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:50:03.921460    4473 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f587fb39-646c-4b6b-8451-4713172f35b0-openshift-service-ca\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:50:03.921467 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:50:03.921474    4473 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f587fb39-646c-4b6b-8451-4713172f35b0-maas-api-tls\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:50:03.921724 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:50:03.921483    4473 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f587fb39-646c-4b6b-8451-4713172f35b0-metrics-certs\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:50:04.310666 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:50:04.310631    4473 generic.go:356] "Generic (PLEG): container finished" podID="f587fb39-646c-4b6b-8451-4713172f35b0" containerID="267adefae37383881736885b9247a42587058927bb323952b3601475157ac31b" exitCode=0
Oct 30 21:50:04.311107 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:50:04.310694    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-gn8v4"
Oct 30 21:50:04.311107 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:50:04.310705    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-gn8v4" event={"ID":"f587fb39-646c-4b6b-8451-4713172f35b0","Type":"ContainerDied","Data":"267adefae37383881736885b9247a42587058927bb323952b3601475157ac31b"}
Oct 30 21:50:04.311107 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:50:04.310771    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-gn8v4" event={"ID":"f587fb39-646c-4b6b-8451-4713172f35b0","Type":"ContainerDied","Data":"0eaa9cf1bc94a1d6e3aab4658bcb8a8d644d2ebbf38865c253f74e775766dbbe"}
Oct 30 21:50:04.311107 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:50:04.310791    4473 scope.go:122] "RemoveContainer" containerID="267adefae37383881736885b9247a42587058927bb323952b3601475157ac31b"
Oct 30 21:50:04.320082 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:50:04.320064    4473 scope.go:122] "RemoveContainer" containerID="267adefae37383881736885b9247a42587058927bb323952b3601475157ac31b"
Oct 30 21:50:04.320370 ip-10-0-140-18 kubenswrapper[4473]: E1030 21:50:04.320351    4473 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"267adefae37383881736885b9247a42587058927bb323952b3601475157ac31b\": container with ID starting with 267adefae37383881736885b9247a42587058927bb323952b3601475157ac31b not found: ID does not exist" containerID="267adefae37383881736885b9247a42587058927bb323952b3601475157ac31b"
Oct 30 21:50:04.320417 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:50:04.320380    4473 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"267adefae37383881736885b9247a42587058927bb323952b3601475157ac31b"} err="failed to get container status \"267adefae37383881736885b9247a42587058927bb323952b3601475157ac31b\": rpc error: code = NotFound desc = could not find container \"267adefae37383881736885b9247a42587058927bb323952b3601475157ac31b\": container with ID starting with 267adefae37383881736885b9247a42587058927bb323952b3601475157ac31b not found: ID does not exist"
Oct 30 21:50:04.478871 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:50:04.478837    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-6d669c7fcd-gn8v4"]
Oct 30 21:50:04.529675 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:50:04.529643    4473 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-6d669c7fcd-gn8v4"]
Oct 30 21:50:06.355009 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:50:06.354976    4473 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f587fb39-646c-4b6b-8451-4713172f35b0" path="/var/lib/kubelet/pods/f587fb39-646c-4b6b-8451-4713172f35b0/volumes"
Oct 30 21:50:08.285418 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:50:08.285387    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-769875db56-wr8rv"
Oct 30 21:50:20.322509 ip-10-0-140-18 kubenswrapper[4473]: E1030 21:50:20.322466    4473 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fncvg" containerName="extractor"
Oct 30 21:50:50.323424 ip-10-0-140-18 kubenswrapper[4473]: E1030 21:50:50.323341    4473 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fncvg" containerName="extractor"
Oct 30 21:51:20.326642 ip-10-0-140-18 kubenswrapper[4473]: E1030 21:51:20.326601    4473 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fncvg" containerName="extractor"
Oct 30 21:51:50.323335 ip-10-0-140-18 kubenswrapper[4473]: E1030 21:51:50.323300    4473 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fncvg" containerName="extractor"
Oct 30 21:51:53.176611 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:51:53.176578    4473 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/echo-simulator-kserve-64b6f8f96c-m6tlf"]
Oct 30 21:51:53.177127 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:51:53.176962    4473 cpu_manager.go:414] "containerMap: removing container" podUID="f587fb39-646c-4b6b-8451-4713172f35b0" containerName="maas-api"
Oct 30 21:51:53.177127 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:51:53.176973    4473 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f587fb39-646c-4b6b-8451-4713172f35b0" containerName="maas-api"
Oct 30 21:51:53.177127 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:51:53.177043    4473 memory_manager.go:365] "RemoveStaleState removing state" podUID="f587fb39-646c-4b6b-8451-4713172f35b0" containerName="maas-api"
Oct 30 21:51:53.179993 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:51:53.179975    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/echo-simulator-kserve-64b6f8f96c-m6tlf"
Oct 30 21:51:53.233308 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:51:53.233281    4473 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Oct 30 21:51:53.233308 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:51:53.233291    4473 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Oct 30 21:51:53.247218 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:51:53.247186    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e9ca7766-41cc-4042-ae52-ce6451f31d4b-tmp-dir\") pod \"echo-simulator-kserve-64b6f8f96c-m6tlf\" (UID: \"e9ca7766-41cc-4042-ae52-ce6451f31d4b\") " pod="llm/echo-simulator-kserve-64b6f8f96c-m6tlf"
Oct 30 21:51:53.247416 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:51:53.247242    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gfvgj\" (UniqueName: \"kubernetes.io/projected/e9ca7766-41cc-4042-ae52-ce6451f31d4b-kube-api-access-gfvgj\") pod \"echo-simulator-kserve-64b6f8f96c-m6tlf\" (UID: \"e9ca7766-41cc-4042-ae52-ce6451f31d4b\") " pod="llm/echo-simulator-kserve-64b6f8f96c-m6tlf"
Oct 30 21:51:53.247416 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:51:53.247345    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e9ca7766-41cc-4042-ae52-ce6451f31d4b-model-cache\") pod \"echo-simulator-kserve-64b6f8f96c-m6tlf\" (UID: \"e9ca7766-41cc-4042-ae52-ce6451f31d4b\") " pod="llm/echo-simulator-kserve-64b6f8f96c-m6tlf"
Oct 30 21:51:53.247532 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:51:53.247435    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e9ca7766-41cc-4042-ae52-ce6451f31d4b-dshm\") pod \"echo-simulator-kserve-64b6f8f96c-m6tlf\" (UID: \"e9ca7766-41cc-4042-ae52-ce6451f31d4b\") " pod="llm/echo-simulator-kserve-64b6f8f96c-m6tlf"
Oct 30 21:51:53.247569 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:51:53.247533    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e9ca7766-41cc-4042-ae52-ce6451f31d4b-tls-certs\") pod \"echo-simulator-kserve-64b6f8f96c-m6tlf\" (UID: \"e9ca7766-41cc-4042-ae52-ce6451f31d4b\") " pod="llm/echo-simulator-kserve-64b6f8f96c-m6tlf"
Oct 30 21:51:53.247569 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:51:53.247563    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e9ca7766-41cc-4042-ae52-ce6451f31d4b-home\") pod \"echo-simulator-kserve-64b6f8f96c-m6tlf\" (UID: \"e9ca7766-41cc-4042-ae52-ce6451f31d4b\") " pod="llm/echo-simulator-kserve-64b6f8f96c-m6tlf"
Oct 30 21:51:53.283937 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:51:53.283886    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"echo-simulator-kserve-self-signed-certs\""
Oct 30 21:51:53.283937 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:51:53.283910    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-rghth\""
Oct 30 21:51:53.285375 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:51:53.285354    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/echo-simulator-kserve-64b6f8f96c-m6tlf"]
Oct 30 21:51:53.348688 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:51:53.348650    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e9ca7766-41cc-4042-ae52-ce6451f31d4b-tmp-dir\") pod \"echo-simulator-kserve-64b6f8f96c-m6tlf\" (UID: \"e9ca7766-41cc-4042-ae52-ce6451f31d4b\") " pod="llm/echo-simulator-kserve-64b6f8f96c-m6tlf"
Oct 30 21:51:53.348893 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:51:53.348721    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gfvgj\" (UniqueName: \"kubernetes.io/projected/e9ca7766-41cc-4042-ae52-ce6451f31d4b-kube-api-access-gfvgj\") pod \"echo-simulator-kserve-64b6f8f96c-m6tlf\" (UID: \"e9ca7766-41cc-4042-ae52-ce6451f31d4b\") " pod="llm/echo-simulator-kserve-64b6f8f96c-m6tlf"
Oct 30 21:51:53.348893 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:51:53.348760    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e9ca7766-41cc-4042-ae52-ce6451f31d4b-model-cache\") pod \"echo-simulator-kserve-64b6f8f96c-m6tlf\" (UID: \"e9ca7766-41cc-4042-ae52-ce6451f31d4b\") " pod="llm/echo-simulator-kserve-64b6f8f96c-m6tlf"
Oct 30 21:51:53.348893 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:51:53.348783    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e9ca7766-41cc-4042-ae52-ce6451f31d4b-dshm\") pod \"echo-simulator-kserve-64b6f8f96c-m6tlf\" (UID: \"e9ca7766-41cc-4042-ae52-ce6451f31d4b\") " pod="llm/echo-simulator-kserve-64b6f8f96c-m6tlf"
Oct 30 21:51:53.348893 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:51:53.348831    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e9ca7766-41cc-4042-ae52-ce6451f31d4b-tls-certs\") pod \"echo-simulator-kserve-64b6f8f96c-m6tlf\" (UID: \"e9ca7766-41cc-4042-ae52-ce6451f31d4b\") " pod="llm/echo-simulator-kserve-64b6f8f96c-m6tlf"
Oct 30 21:51:53.348893 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:51:53.348855    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e9ca7766-41cc-4042-ae52-ce6451f31d4b-home\") pod \"echo-simulator-kserve-64b6f8f96c-m6tlf\" (UID: \"e9ca7766-41cc-4042-ae52-ce6451f31d4b\") " pod="llm/echo-simulator-kserve-64b6f8f96c-m6tlf"
Oct 30 21:51:53.349181 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:51:53.349144    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e9ca7766-41cc-4042-ae52-ce6451f31d4b-tmp-dir\") pod \"echo-simulator-kserve-64b6f8f96c-m6tlf\" (UID: \"e9ca7766-41cc-4042-ae52-ce6451f31d4b\") " pod="llm/echo-simulator-kserve-64b6f8f96c-m6tlf"
Oct 30 21:51:53.349245 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:51:53.349183    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e9ca7766-41cc-4042-ae52-ce6451f31d4b-model-cache\") pod \"echo-simulator-kserve-64b6f8f96c-m6tlf\" (UID: \"e9ca7766-41cc-4042-ae52-ce6451f31d4b\") " pod="llm/echo-simulator-kserve-64b6f8f96c-m6tlf"
Oct 30 21:51:53.349313 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:51:53.349291    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e9ca7766-41cc-4042-ae52-ce6451f31d4b-home\") pod \"echo-simulator-kserve-64b6f8f96c-m6tlf\" (UID: \"e9ca7766-41cc-4042-ae52-ce6451f31d4b\") " pod="llm/echo-simulator-kserve-64b6f8f96c-m6tlf"
Oct 30 21:51:53.351242 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:51:53.351214    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e9ca7766-41cc-4042-ae52-ce6451f31d4b-dshm\") pod \"echo-simulator-kserve-64b6f8f96c-m6tlf\" (UID: \"e9ca7766-41cc-4042-ae52-ce6451f31d4b\") " pod="llm/echo-simulator-kserve-64b6f8f96c-m6tlf"
Oct 30 21:51:53.351507 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:51:53.351490    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e9ca7766-41cc-4042-ae52-ce6451f31d4b-tls-certs\") pod \"echo-simulator-kserve-64b6f8f96c-m6tlf\" (UID: \"e9ca7766-41cc-4042-ae52-ce6451f31d4b\") " pod="llm/echo-simulator-kserve-64b6f8f96c-m6tlf"
Oct 30 21:51:53.407316 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:51:53.407280    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gfvgj\" (UniqueName: \"kubernetes.io/projected/e9ca7766-41cc-4042-ae52-ce6451f31d4b-kube-api-access-gfvgj\") pod \"echo-simulator-kserve-64b6f8f96c-m6tlf\" (UID: \"e9ca7766-41cc-4042-ae52-ce6451f31d4b\") " pod="llm/echo-simulator-kserve-64b6f8f96c-m6tlf"
Oct 30 21:51:53.493050 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:51:53.492963    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/echo-simulator-kserve-64b6f8f96c-m6tlf"
Oct 30 21:51:53.871698 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:51:53.871627    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/echo-simulator-kserve-64b6f8f96c-m6tlf"]
Oct 30 21:51:53.876412 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:51:53.876382    4473 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9ca7766_41cc_4042_ae52_ce6451f31d4b.slice/crio-75aa466c7c721ade8f71fc36628b7284cafce3b5080ce75842a19cc31079a973 WatchSource:0}: Error finding container 75aa466c7c721ade8f71fc36628b7284cafce3b5080ce75842a19cc31079a973: Status 404 returned error can't find the container with id 75aa466c7c721ade8f71fc36628b7284cafce3b5080ce75842a19cc31079a973
Oct 30 21:51:54.777513 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:51:54.777471    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/echo-simulator-kserve-64b6f8f96c-m6tlf" event={"ID":"e9ca7766-41cc-4042-ae52-ce6451f31d4b","Type":"ContainerStarted","Data":"75aa466c7c721ade8f71fc36628b7284cafce3b5080ce75842a19cc31079a973"}
Oct 30 21:51:56.788112 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:51:56.788072    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/echo-simulator-kserve-64b6f8f96c-m6tlf" event={"ID":"e9ca7766-41cc-4042-ae52-ce6451f31d4b","Type":"ContainerStarted","Data":"6fcee39d2a201e6d93d8d91cef70b0b734cfc474ddbb8746f1936f12b31d1f1b"}
Oct 30 21:51:56.906885 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:51:56.906832    4473 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/echo-simulator-kserve-64b6f8f96c-m6tlf" podStartSLOduration=1.862271903 podStartE2EDuration="3.906819174s" totalImagesPullingTime="2.044547271s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:51:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:51:56.906058939 +0000 UTC m=+562.679427427" watchObservedRunningTime="2026-10-30 21:51:56.906819174 +0000 UTC m=+562.680187661"
Oct 30 21:51:57.440992 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:51:57.440958    4473 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-embedding-simulated-kserve-67bf594cd6-mgkz8"]
Oct 30 21:51:57.444361 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:51:57.444340    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-mgkz8"
Oct 30 21:51:57.494021 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:51:57.493992    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/echo-simulator-kserve-64b6f8f96c-m6tlf"
Oct 30 21:51:57.547493 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:51:57.547452    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-embedding-simulated-kserve-self-signed-certs\""
Oct 30 21:51:57.548896 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:51:57.548871    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-67bf594cd6-mgkz8"]
Oct 30 21:51:57.590154 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:51:57.590119    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d3644b22-59fe-41ab-99b5-227e15669a15-tls-certs\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-mgkz8\" (UID: \"d3644b22-59fe-41ab-99b5-227e15669a15\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-mgkz8"
Oct 30 21:51:57.590349 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:51:57.590189    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d3644b22-59fe-41ab-99b5-227e15669a15-home\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-mgkz8\" (UID: \"d3644b22-59fe-41ab-99b5-227e15669a15\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-mgkz8"
Oct 30 21:51:57.590349 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:51:57.590251    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d3644b22-59fe-41ab-99b5-227e15669a15-dshm\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-mgkz8\" (UID: \"d3644b22-59fe-41ab-99b5-227e15669a15\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-mgkz8"
Oct 30 21:51:57.590560 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:51:57.590360    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d3644b22-59fe-41ab-99b5-227e15669a15-model-cache\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-mgkz8\" (UID: \"d3644b22-59fe-41ab-99b5-227e15669a15\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-mgkz8"
Oct 30 21:51:57.590560 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:51:57.590438    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d3644b22-59fe-41ab-99b5-227e15669a15-tmp-dir\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-mgkz8\" (UID: \"d3644b22-59fe-41ab-99b5-227e15669a15\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-mgkz8"
Oct 30 21:51:57.590560 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:51:57.590486    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rrh4f\" (UniqueName: \"kubernetes.io/projected/d3644b22-59fe-41ab-99b5-227e15669a15-kube-api-access-rrh4f\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-mgkz8\" (UID: \"d3644b22-59fe-41ab-99b5-227e15669a15\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-mgkz8"
Oct 30 21:51:57.691761 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:51:57.691671    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d3644b22-59fe-41ab-99b5-227e15669a15-tls-certs\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-mgkz8\" (UID: \"d3644b22-59fe-41ab-99b5-227e15669a15\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-mgkz8"
Oct 30 21:51:57.691761 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:51:57.691722    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d3644b22-59fe-41ab-99b5-227e15669a15-home\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-mgkz8\" (UID: \"d3644b22-59fe-41ab-99b5-227e15669a15\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-mgkz8"
Oct 30 21:51:57.691761 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:51:57.691743    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d3644b22-59fe-41ab-99b5-227e15669a15-dshm\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-mgkz8\" (UID: \"d3644b22-59fe-41ab-99b5-227e15669a15\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-mgkz8"
Oct 30 21:51:57.692099 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:51:57.691796    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d3644b22-59fe-41ab-99b5-227e15669a15-model-cache\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-mgkz8\" (UID: \"d3644b22-59fe-41ab-99b5-227e15669a15\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-mgkz8"
Oct 30 21:51:57.692099 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:51:57.691951    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d3644b22-59fe-41ab-99b5-227e15669a15-tmp-dir\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-mgkz8\" (UID: \"d3644b22-59fe-41ab-99b5-227e15669a15\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-mgkz8"
Oct 30 21:51:57.692099 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:51:57.691979    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rrh4f\" (UniqueName: \"kubernetes.io/projected/d3644b22-59fe-41ab-99b5-227e15669a15-kube-api-access-rrh4f\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-mgkz8\" (UID: \"d3644b22-59fe-41ab-99b5-227e15669a15\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-mgkz8"
Oct 30 21:51:57.692257 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:51:57.692128    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d3644b22-59fe-41ab-99b5-227e15669a15-home\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-mgkz8\" (UID: \"d3644b22-59fe-41ab-99b5-227e15669a15\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-mgkz8"
Oct 30 21:51:57.692257 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:51:57.692190    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d3644b22-59fe-41ab-99b5-227e15669a15-model-cache\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-mgkz8\" (UID: \"d3644b22-59fe-41ab-99b5-227e15669a15\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-mgkz8"
Oct 30 21:51:57.692326 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:51:57.692301    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d3644b22-59fe-41ab-99b5-227e15669a15-tmp-dir\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-mgkz8\" (UID: \"d3644b22-59fe-41ab-99b5-227e15669a15\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-mgkz8"
Oct 30 21:51:57.694317 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:51:57.694290    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d3644b22-59fe-41ab-99b5-227e15669a15-dshm\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-mgkz8\" (UID: \"d3644b22-59fe-41ab-99b5-227e15669a15\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-mgkz8"
Oct 30 21:51:57.694493 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:51:57.694476    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d3644b22-59fe-41ab-99b5-227e15669a15-tls-certs\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-mgkz8\" (UID: \"d3644b22-59fe-41ab-99b5-227e15669a15\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-mgkz8"
Oct 30 21:51:57.755633 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:51:57.755600    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rrh4f\" (UniqueName: \"kubernetes.io/projected/d3644b22-59fe-41ab-99b5-227e15669a15-kube-api-access-rrh4f\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-mgkz8\" (UID: \"d3644b22-59fe-41ab-99b5-227e15669a15\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-mgkz8"
Oct 30 21:51:57.756333 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:51:57.756313    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-mgkz8"
Oct 30 21:51:58.177270 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:51:58.177244    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-67bf594cd6-mgkz8"]
Oct 30 21:51:58.179247 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:51:58.179221    4473 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3644b22_59fe_41ab_99b5_227e15669a15.slice/crio-aa08033df93ddfed9493029ba3ac67f89c94e5c4b048e542055427fd2fe7a413 WatchSource:0}: Error finding container aa08033df93ddfed9493029ba3ac67f89c94e5c4b048e542055427fd2fe7a413: Status 404 returned error can't find the container with id aa08033df93ddfed9493029ba3ac67f89c94e5c4b048e542055427fd2fe7a413
Oct 30 21:51:58.797813 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:51:58.797771    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-mgkz8" event={"ID":"d3644b22-59fe-41ab-99b5-227e15669a15","Type":"ContainerStarted","Data":"aa08033df93ddfed9493029ba3ac67f89c94e5c4b048e542055427fd2fe7a413"}
Oct 30 21:52:00.508362 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:00.508331    4473 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-69b588897d-fkv7q"]
Oct 30 21:52:00.511976 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:00.511957    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-fkv7q"
Oct 30 21:52:00.618131 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:00.618095    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Oct 30 21:52:00.619643 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:00.619618    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-69b588897d-fkv7q"]
Oct 30 21:52:00.622463 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:00.622442    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8265cdcf-eaa9-4e4f-8250-915df95739c2-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-fkv7q\" (UID: \"8265cdcf-eaa9-4e4f-8250-915df95739c2\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-fkv7q"
Oct 30 21:52:00.622522 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:00.622474    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8265cdcf-eaa9-4e4f-8250-915df95739c2-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-fkv7q\" (UID: \"8265cdcf-eaa9-4e4f-8250-915df95739c2\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-fkv7q"
Oct 30 21:52:00.622522 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:00.622495    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8265cdcf-eaa9-4e4f-8250-915df95739c2-model-cache\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-fkv7q\" (UID: \"8265cdcf-eaa9-4e4f-8250-915df95739c2\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-fkv7q"
Oct 30 21:52:00.622656 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:00.622639    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8265cdcf-eaa9-4e4f-8250-915df95739c2-home\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-fkv7q\" (UID: \"8265cdcf-eaa9-4e4f-8250-915df95739c2\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-fkv7q"
Oct 30 21:52:00.622725 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:00.622709    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8265cdcf-eaa9-4e4f-8250-915df95739c2-dshm\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-fkv7q\" (UID: \"8265cdcf-eaa9-4e4f-8250-915df95739c2\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-fkv7q"
Oct 30 21:52:00.622780 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:00.622759    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bmdsk\" (UniqueName: \"kubernetes.io/projected/8265cdcf-eaa9-4e4f-8250-915df95739c2-kube-api-access-bmdsk\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-fkv7q\" (UID: \"8265cdcf-eaa9-4e4f-8250-915df95739c2\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-fkv7q"
Oct 30 21:52:00.724149 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:00.724115    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8265cdcf-eaa9-4e4f-8250-915df95739c2-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-fkv7q\" (UID: \"8265cdcf-eaa9-4e4f-8250-915df95739c2\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-fkv7q"
Oct 30 21:52:00.724149 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:00.724154    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8265cdcf-eaa9-4e4f-8250-915df95739c2-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-fkv7q\" (UID: \"8265cdcf-eaa9-4e4f-8250-915df95739c2\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-fkv7q"
Oct 30 21:52:00.724350 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:00.724174    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8265cdcf-eaa9-4e4f-8250-915df95739c2-model-cache\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-fkv7q\" (UID: \"8265cdcf-eaa9-4e4f-8250-915df95739c2\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-fkv7q"
Oct 30 21:52:00.724350 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:00.724211    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8265cdcf-eaa9-4e4f-8250-915df95739c2-home\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-fkv7q\" (UID: \"8265cdcf-eaa9-4e4f-8250-915df95739c2\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-fkv7q"
Oct 30 21:52:00.724350 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:00.724233    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8265cdcf-eaa9-4e4f-8250-915df95739c2-dshm\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-fkv7q\" (UID: \"8265cdcf-eaa9-4e4f-8250-915df95739c2\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-fkv7q"
Oct 30 21:52:00.724350 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:00.724264    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bmdsk\" (UniqueName: \"kubernetes.io/projected/8265cdcf-eaa9-4e4f-8250-915df95739c2-kube-api-access-bmdsk\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-fkv7q\" (UID: \"8265cdcf-eaa9-4e4f-8250-915df95739c2\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-fkv7q"
Oct 30 21:52:00.724642 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:00.724617    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8265cdcf-eaa9-4e4f-8250-915df95739c2-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-fkv7q\" (UID: \"8265cdcf-eaa9-4e4f-8250-915df95739c2\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-fkv7q"
Oct 30 21:52:00.724699 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:00.724641    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8265cdcf-eaa9-4e4f-8250-915df95739c2-model-cache\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-fkv7q\" (UID: \"8265cdcf-eaa9-4e4f-8250-915df95739c2\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-fkv7q"
Oct 30 21:52:00.724699 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:00.724661    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8265cdcf-eaa9-4e4f-8250-915df95739c2-home\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-fkv7q\" (UID: \"8265cdcf-eaa9-4e4f-8250-915df95739c2\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-fkv7q"
Oct 30 21:52:00.726747 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:00.726719    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8265cdcf-eaa9-4e4f-8250-915df95739c2-dshm\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-fkv7q\" (UID: \"8265cdcf-eaa9-4e4f-8250-915df95739c2\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-fkv7q"
Oct 30 21:52:00.726956 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:00.726913    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8265cdcf-eaa9-4e4f-8250-915df95739c2-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-fkv7q\" (UID: \"8265cdcf-eaa9-4e4f-8250-915df95739c2\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-fkv7q"
Oct 30 21:52:00.783265 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:00.783183    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bmdsk\" (UniqueName: \"kubernetes.io/projected/8265cdcf-eaa9-4e4f-8250-915df95739c2-kube-api-access-bmdsk\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-fkv7q\" (UID: \"8265cdcf-eaa9-4e4f-8250-915df95739c2\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-fkv7q"
Oct 30 21:52:00.808224 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:00.808190    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-mgkz8" event={"ID":"d3644b22-59fe-41ab-99b5-227e15669a15","Type":"ContainerStarted","Data":"fbbd95a2a3ba4e76f414479194467e16d4bdaf870bddf68b2d44c620b33bffe4"}
Oct 30 21:52:00.824666 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:00.824639    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-fkv7q"
Oct 30 21:52:00.928607 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:00.928559    4473 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-mgkz8" podStartSLOduration=2.333153441 podStartE2EDuration="3.928543891s" totalImagesPullingTime="1.59539045s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:51:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:52:00.927595848 +0000 UTC m=+566.700964336" watchObservedRunningTime="2026-10-30 21:52:00.928543891 +0000 UTC m=+566.701912374"
Oct 30 21:52:01.202879 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:01.202855    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-69b588897d-fkv7q"]
Oct 30 21:52:01.205070 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:52:01.205033    4473 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8265cdcf_eaa9_4e4f_8250_915df95739c2.slice/crio-d1ab502e06e7cc4fb219244342697fe54dbb662f561617a18032ad8e74ff3780 WatchSource:0}: Error finding container d1ab502e06e7cc4fb219244342697fe54dbb662f561617a18032ad8e74ff3780: Status 404 returned error can't find the container with id d1ab502e06e7cc4fb219244342697fe54dbb662f561617a18032ad8e74ff3780
Oct 30 21:52:01.757211 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:01.757179    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-mgkz8"
Oct 30 21:52:01.814184 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:01.814143    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-fkv7q" event={"ID":"8265cdcf-eaa9-4e4f-8250-915df95739c2","Type":"ContainerStarted","Data":"d1ab502e06e7cc4fb219244342697fe54dbb662f561617a18032ad8e74ff3780"}
Oct 30 21:52:02.819913 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:02.819875    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-fkv7q" event={"ID":"8265cdcf-eaa9-4e4f-8250-915df95739c2","Type":"ContainerStarted","Data":"6e4926e5c6849753afd7906930a702bd4e6668a2b579fdd7ce416c7311b93891"}
Oct 30 21:52:02.825320 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:02.825300    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-fkv7q"
Oct 30 21:52:02.939212 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:02.939155    4473 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-fkv7q" podStartSLOduration=2.244776125 podStartE2EDuration="2.939140007s" totalImagesPullingTime="694.363882ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:52:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:52:02.938303792 +0000 UTC m=+568.711672279" watchObservedRunningTime="2026-10-30 21:52:02.939140007 +0000 UTC m=+568.712508493"
Oct 30 21:52:03.493984 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:03.493947    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/echo-simulator-kserve-64b6f8f96c-m6tlf"
Oct 30 21:52:03.506684 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:03.506655    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/echo-simulator-kserve-64b6f8f96c-m6tlf"
Oct 30 21:52:03.836056 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:03.835979    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/echo-simulator-kserve-64b6f8f96c-m6tlf"
Oct 30 21:52:07.757181 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:07.757131    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-mgkz8"
Oct 30 21:52:07.769888 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:07.769860    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-mgkz8"
Oct 30 21:52:07.855178 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:07.855146    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-mgkz8"
Oct 30 21:52:10.825229 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:10.825179    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-fkv7q"
Oct 30 21:52:10.843483 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:10.843452    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-fkv7q"
Oct 30 21:52:10.871085 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:10.871058    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-fkv7q"
Oct 30 21:52:17.732075 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:17.731997    4473 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-cbd4f5586-5g7dx"]
Oct 30 21:52:17.735272 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:17.735252    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-5g7dx"
Oct 30 21:52:17.769966 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:17.769908    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/da982b32-3d1a-4766-b9d4-6804b0684cd3-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-5g7dx\" (UID: \"da982b32-3d1a-4766-b9d4-6804b0684cd3\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-5g7dx"
Oct 30 21:52:17.770116 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:17.770043    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/da982b32-3d1a-4766-b9d4-6804b0684cd3-model-cache\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-5g7dx\" (UID: \"da982b32-3d1a-4766-b9d4-6804b0684cd3\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-5g7dx"
Oct 30 21:52:17.770163 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:17.770114    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/da982b32-3d1a-4766-b9d4-6804b0684cd3-home\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-5g7dx\" (UID: \"da982b32-3d1a-4766-b9d4-6804b0684cd3\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-5g7dx"
Oct 30 21:52:17.770163 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:17.770147    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cztlc\" (UniqueName: \"kubernetes.io/projected/da982b32-3d1a-4766-b9d4-6804b0684cd3-kube-api-access-cztlc\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-5g7dx\" (UID: \"da982b32-3d1a-4766-b9d4-6804b0684cd3\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-5g7dx"
Oct 30 21:52:17.770256 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:17.770234    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/da982b32-3d1a-4766-b9d4-6804b0684cd3-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-5g7dx\" (UID: \"da982b32-3d1a-4766-b9d4-6804b0684cd3\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-5g7dx"
Oct 30 21:52:17.770343 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:17.770323    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/da982b32-3d1a-4766-b9d4-6804b0684cd3-dshm\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-5g7dx\" (UID: \"da982b32-3d1a-4766-b9d4-6804b0684cd3\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-5g7dx"
Oct 30 21:52:17.838380 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:17.838350    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Oct 30 21:52:17.840173 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:17.840141    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-cbd4f5586-5g7dx"]
Oct 30 21:52:17.871211 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:17.871181    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/da982b32-3d1a-4766-b9d4-6804b0684cd3-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-5g7dx\" (UID: \"da982b32-3d1a-4766-b9d4-6804b0684cd3\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-5g7dx"
Oct 30 21:52:17.871211 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:17.871215    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/da982b32-3d1a-4766-b9d4-6804b0684cd3-model-cache\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-5g7dx\" (UID: \"da982b32-3d1a-4766-b9d4-6804b0684cd3\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-5g7dx"
Oct 30 21:52:17.871484 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:17.871245    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/da982b32-3d1a-4766-b9d4-6804b0684cd3-home\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-5g7dx\" (UID: \"da982b32-3d1a-4766-b9d4-6804b0684cd3\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-5g7dx"
Oct 30 21:52:17.871484 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:17.871270    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cztlc\" (UniqueName: \"kubernetes.io/projected/da982b32-3d1a-4766-b9d4-6804b0684cd3-kube-api-access-cztlc\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-5g7dx\" (UID: \"da982b32-3d1a-4766-b9d4-6804b0684cd3\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-5g7dx"
Oct 30 21:52:17.871484 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:17.871352    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/da982b32-3d1a-4766-b9d4-6804b0684cd3-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-5g7dx\" (UID: \"da982b32-3d1a-4766-b9d4-6804b0684cd3\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-5g7dx"
Oct 30 21:52:17.871484 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:17.871395    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/da982b32-3d1a-4766-b9d4-6804b0684cd3-dshm\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-5g7dx\" (UID: \"da982b32-3d1a-4766-b9d4-6804b0684cd3\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-5g7dx"
Oct 30 21:52:17.871717 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:17.871699    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/da982b32-3d1a-4766-b9d4-6804b0684cd3-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-5g7dx\" (UID: \"da982b32-3d1a-4766-b9d4-6804b0684cd3\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-5g7dx"
Oct 30 21:52:17.871786 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:17.871742    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/da982b32-3d1a-4766-b9d4-6804b0684cd3-home\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-5g7dx\" (UID: \"da982b32-3d1a-4766-b9d4-6804b0684cd3\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-5g7dx"
Oct 30 21:52:17.871786 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:17.871758    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/da982b32-3d1a-4766-b9d4-6804b0684cd3-model-cache\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-5g7dx\" (UID: \"da982b32-3d1a-4766-b9d4-6804b0684cd3\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-5g7dx"
Oct 30 21:52:17.873996 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:17.873971    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/da982b32-3d1a-4766-b9d4-6804b0684cd3-dshm\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-5g7dx\" (UID: \"da982b32-3d1a-4766-b9d4-6804b0684cd3\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-5g7dx"
Oct 30 21:52:17.874536 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:17.874513    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/da982b32-3d1a-4766-b9d4-6804b0684cd3-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-5g7dx\" (UID: \"da982b32-3d1a-4766-b9d4-6804b0684cd3\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-5g7dx"
Oct 30 21:52:17.928815 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:17.928780    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cztlc\" (UniqueName: \"kubernetes.io/projected/da982b32-3d1a-4766-b9d4-6804b0684cd3-kube-api-access-cztlc\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-5g7dx\" (UID: \"da982b32-3d1a-4766-b9d4-6804b0684cd3\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-5g7dx"
Oct 30 21:52:18.048153 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:18.048067    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-5g7dx"
Oct 30 21:52:18.424635 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:18.424611    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-cbd4f5586-5g7dx"]
Oct 30 21:52:18.426620 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:52:18.426591    4473 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda982b32_3d1a_4766_b9d4_6804b0684cd3.slice/crio-c2f8b9bb622eb5f867b8fc8a40768e66353cee1d709a7695a819abf71412dfaf WatchSource:0}: Error finding container c2f8b9bb622eb5f867b8fc8a40768e66353cee1d709a7695a819abf71412dfaf: Status 404 returned error can't find the container with id c2f8b9bb622eb5f867b8fc8a40768e66353cee1d709a7695a819abf71412dfaf
Oct 30 21:52:18.898571 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:18.898534    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-5g7dx" event={"ID":"da982b32-3d1a-4766-b9d4-6804b0684cd3","Type":"ContainerStarted","Data":"c2f8b9bb622eb5f867b8fc8a40768e66353cee1d709a7695a819abf71412dfaf"}
Oct 30 21:52:19.904589 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:19.904545    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-5g7dx" event={"ID":"da982b32-3d1a-4766-b9d4-6804b0684cd3","Type":"ContainerStarted","Data":"f26fbd3e61c38081646c42881694f872b8257d6d16b8fa7962c15205d82c9fa2"}
Oct 30 21:52:20.020637 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:20.020573    4473 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-5g7dx" podStartSLOduration=2.429618052 podStartE2EDuration="3.020557688s" totalImagesPullingTime="590.939636ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:52:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:52:20.020347983 +0000 UTC m=+585.793716471" watchObservedRunningTime="2026-10-30 21:52:20.020557688 +0000 UTC m=+585.793926169"
Oct 30 21:52:20.048370 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:20.048334    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-5g7dx"
Oct 30 21:52:20.324969 ip-10-0-140-18 kubenswrapper[4473]: E1030 21:52:20.324867    4473 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fncvg" containerName="extractor"
Oct 30 21:52:28.048975 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:28.048853    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-5g7dx"
Oct 30 21:52:28.061688 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:28.061659    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-5g7dx"
Oct 30 21:52:28.958105 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:28.958074    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-5g7dx"
Oct 30 21:52:30.257483 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:30.257447    4473 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-ggdbq"]
Oct 30 21:52:30.265451 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:30.265424    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-ggdbq"
Oct 30 21:52:30.324064 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:30.324029    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-v2z7t\""
Oct 30 21:52:30.373407 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:30.373370    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-ggdbq"]
Oct 30 21:52:30.380687 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:30.380648    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mk8hq\" (UniqueName: \"kubernetes.io/projected/5cbe0443-8a90-4133-9f81-42bdca7d7e1a-kube-api-access-mk8hq\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-ggdbq\" (UID: \"5cbe0443-8a90-4133-9f81-42bdca7d7e1a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-ggdbq"
Oct 30 21:52:30.380869 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:30.380710    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5cbe0443-8a90-4133-9f81-42bdca7d7e1a-istio-data\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-ggdbq\" (UID: \"5cbe0443-8a90-4133-9f81-42bdca7d7e1a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-ggdbq"
Oct 30 21:52:30.380869 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:30.380821    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5cbe0443-8a90-4133-9f81-42bdca7d7e1a-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-ggdbq\" (UID: \"5cbe0443-8a90-4133-9f81-42bdca7d7e1a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-ggdbq"
Oct 30 21:52:30.381003 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:30.380885    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5cbe0443-8a90-4133-9f81-42bdca7d7e1a-workload-certs\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-ggdbq\" (UID: \"5cbe0443-8a90-4133-9f81-42bdca7d7e1a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-ggdbq"
Oct 30 21:52:30.381042 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:30.381008    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5cbe0443-8a90-4133-9f81-42bdca7d7e1a-istio-token\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-ggdbq\" (UID: \"5cbe0443-8a90-4133-9f81-42bdca7d7e1a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-ggdbq"
Oct 30 21:52:30.381097 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:30.381083    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5cbe0443-8a90-4133-9f81-42bdca7d7e1a-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-ggdbq\" (UID: \"5cbe0443-8a90-4133-9f81-42bdca7d7e1a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-ggdbq"
Oct 30 21:52:30.381202 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:30.381180    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5cbe0443-8a90-4133-9f81-42bdca7d7e1a-workload-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-ggdbq\" (UID: \"5cbe0443-8a90-4133-9f81-42bdca7d7e1a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-ggdbq"
Oct 30 21:52:30.381261 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:30.381241    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5cbe0443-8a90-4133-9f81-42bdca7d7e1a-istio-envoy\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-ggdbq\" (UID: \"5cbe0443-8a90-4133-9f81-42bdca7d7e1a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-ggdbq"
Oct 30 21:52:30.381343 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:30.381326    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5cbe0443-8a90-4133-9f81-42bdca7d7e1a-credential-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-ggdbq\" (UID: \"5cbe0443-8a90-4133-9f81-42bdca7d7e1a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-ggdbq"
Oct 30 21:52:30.481822 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:30.481785    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5cbe0443-8a90-4133-9f81-42bdca7d7e1a-credential-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-ggdbq\" (UID: \"5cbe0443-8a90-4133-9f81-42bdca7d7e1a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-ggdbq"
Oct 30 21:52:30.482067 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:30.481831    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mk8hq\" (UniqueName: \"kubernetes.io/projected/5cbe0443-8a90-4133-9f81-42bdca7d7e1a-kube-api-access-mk8hq\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-ggdbq\" (UID: \"5cbe0443-8a90-4133-9f81-42bdca7d7e1a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-ggdbq"
Oct 30 21:52:30.482067 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:30.481991    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5cbe0443-8a90-4133-9f81-42bdca7d7e1a-istio-data\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-ggdbq\" (UID: \"5cbe0443-8a90-4133-9f81-42bdca7d7e1a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-ggdbq"
Oct 30 21:52:30.482067 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:30.482056    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5cbe0443-8a90-4133-9f81-42bdca7d7e1a-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-ggdbq\" (UID: \"5cbe0443-8a90-4133-9f81-42bdca7d7e1a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-ggdbq"
Oct 30 21:52:30.482250 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:30.482106    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5cbe0443-8a90-4133-9f81-42bdca7d7e1a-workload-certs\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-ggdbq\" (UID: \"5cbe0443-8a90-4133-9f81-42bdca7d7e1a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-ggdbq"
Oct 30 21:52:30.482250 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:30.482233    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5cbe0443-8a90-4133-9f81-42bdca7d7e1a-istio-token\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-ggdbq\" (UID: \"5cbe0443-8a90-4133-9f81-42bdca7d7e1a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-ggdbq"
Oct 30 21:52:30.482357 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:30.482294    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5cbe0443-8a90-4133-9f81-42bdca7d7e1a-credential-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-ggdbq\" (UID: \"5cbe0443-8a90-4133-9f81-42bdca7d7e1a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-ggdbq"
Oct 30 21:52:30.482357 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:30.482323    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5cbe0443-8a90-4133-9f81-42bdca7d7e1a-istio-data\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-ggdbq\" (UID: \"5cbe0443-8a90-4133-9f81-42bdca7d7e1a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-ggdbq"
Oct 30 21:52:30.482357 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:30.482337    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5cbe0443-8a90-4133-9f81-42bdca7d7e1a-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-ggdbq\" (UID: \"5cbe0443-8a90-4133-9f81-42bdca7d7e1a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-ggdbq"
Oct 30 21:52:30.482501 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:30.482390    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5cbe0443-8a90-4133-9f81-42bdca7d7e1a-workload-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-ggdbq\" (UID: \"5cbe0443-8a90-4133-9f81-42bdca7d7e1a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-ggdbq"
Oct 30 21:52:30.482501 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:30.482421    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5cbe0443-8a90-4133-9f81-42bdca7d7e1a-istio-envoy\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-ggdbq\" (UID: \"5cbe0443-8a90-4133-9f81-42bdca7d7e1a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-ggdbq"
Oct 30 21:52:30.482501 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:30.482429    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5cbe0443-8a90-4133-9f81-42bdca7d7e1a-workload-certs\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-ggdbq\" (UID: \"5cbe0443-8a90-4133-9f81-42bdca7d7e1a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-ggdbq"
Oct 30 21:52:30.482670 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:30.482647    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5cbe0443-8a90-4133-9f81-42bdca7d7e1a-workload-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-ggdbq\" (UID: \"5cbe0443-8a90-4133-9f81-42bdca7d7e1a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-ggdbq"
Oct 30 21:52:30.482786 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:30.482763    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5cbe0443-8a90-4133-9f81-42bdca7d7e1a-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-ggdbq\" (UID: \"5cbe0443-8a90-4133-9f81-42bdca7d7e1a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-ggdbq"
Oct 30 21:52:30.484682 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:30.484660    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5cbe0443-8a90-4133-9f81-42bdca7d7e1a-istio-envoy\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-ggdbq\" (UID: \"5cbe0443-8a90-4133-9f81-42bdca7d7e1a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-ggdbq"
Oct 30 21:52:30.484880 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:30.484864    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5cbe0443-8a90-4133-9f81-42bdca7d7e1a-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-ggdbq\" (UID: \"5cbe0443-8a90-4133-9f81-42bdca7d7e1a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-ggdbq"
Oct 30 21:52:30.556865 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:30.556781    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5cbe0443-8a90-4133-9f81-42bdca7d7e1a-istio-token\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-ggdbq\" (UID: \"5cbe0443-8a90-4133-9f81-42bdca7d7e1a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-ggdbq"
Oct 30 21:52:30.561622 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:30.561595    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mk8hq\" (UniqueName: \"kubernetes.io/projected/5cbe0443-8a90-4133-9f81-42bdca7d7e1a-kube-api-access-mk8hq\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-ggdbq\" (UID: \"5cbe0443-8a90-4133-9f81-42bdca7d7e1a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-ggdbq"
Oct 30 21:52:30.579018 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:30.578982    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-ggdbq"
Oct 30 21:52:31.071201 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:31.071180    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-ggdbq"]
Oct 30 21:52:31.073439 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:52:31.073406    4473 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5cbe0443_8a90_4133_9f81_42bdca7d7e1a.slice/crio-d692fb8df2938062fee68448137ba0cc683c7a673ce8e0355bf30b3fc2a0633f WatchSource:0}: Error finding container d692fb8df2938062fee68448137ba0cc683c7a673ce8e0355bf30b3fc2a0633f: Status 404 returned error can't find the container with id d692fb8df2938062fee68448137ba0cc683c7a673ce8e0355bf30b3fc2a0633f
Oct 30 21:52:31.961133 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:31.961095    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-ggdbq" event={"ID":"5cbe0443-8a90-4133-9f81-42bdca7d7e1a","Type":"ContainerStarted","Data":"d692fb8df2938062fee68448137ba0cc683c7a673ce8e0355bf30b3fc2a0633f"}
Oct 30 21:52:34.572460 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:34.572414    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:52:34.572841 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:34.572492    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:52:34.572841 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:34.572528    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:52:34.978528 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:34.978488    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-ggdbq" event={"ID":"5cbe0443-8a90-4133-9f81-42bdca7d7e1a","Type":"ContainerStarted","Data":"d7e78de214d176ed3eeb9b5fca8a084d718ffe8ee7c06ba3a45d29a724e4b321"}
Oct 30 21:52:35.200179 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:35.200127    4473 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-ggdbq" podStartSLOduration=1.704104322 podStartE2EDuration="5.200112578s" totalImagesPullingTime="3.496008256s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:52:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:52:35.150403121 +0000 UTC m=+600.923771625" watchObservedRunningTime="2026-10-30 21:52:35.200112578 +0000 UTC m=+600.973481064"
Oct 30 21:52:35.579448 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:35.579412    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-ggdbq"
Oct 30 21:52:35.580960 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:35.580912    4473 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-ggdbq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body=
Oct 30 21:52:35.581023 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:35.580985    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-ggdbq" podUID="5cbe0443-8a90-4133-9f81-42bdca7d7e1a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused"
Oct 30 21:52:36.580378 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:36.580339    4473 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-ggdbq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body=
Oct 30 21:52:36.580798 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:36.580397    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-ggdbq" podUID="5cbe0443-8a90-4133-9f81-42bdca7d7e1a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused"
Oct 30 21:52:37.579836 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:37.579797    4473 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-ggdbq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body=
Oct 30 21:52:37.580061 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:37.579852    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-ggdbq" podUID="5cbe0443-8a90-4133-9f81-42bdca7d7e1a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused"
Oct 30 21:52:38.580323 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:38.580287    4473 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-ggdbq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body=
Oct 30 21:52:38.580768 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:38.580352    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-ggdbq" podUID="5cbe0443-8a90-4133-9f81-42bdca7d7e1a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused"
Oct 30 21:52:39.580143 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:39.580102    4473 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-ggdbq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body=
Oct 30 21:52:39.580346 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:39.580159    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-ggdbq" podUID="5cbe0443-8a90-4133-9f81-42bdca7d7e1a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused"
Oct 30 21:52:40.580448 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:40.580390    4473 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-ggdbq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body=
Oct 30 21:52:40.580894 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:40.580480    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-ggdbq" podUID="5cbe0443-8a90-4133-9f81-42bdca7d7e1a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused"
Oct 30 21:52:41.580169 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:41.580126    4473 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-ggdbq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body=
Oct 30 21:52:41.580352 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:41.580202    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-ggdbq" podUID="5cbe0443-8a90-4133-9f81-42bdca7d7e1a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused"
Oct 30 21:52:42.878114 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:42.878075    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-ggdbq"
Oct 30 21:52:42.878584 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:42.878380    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-ggdbq"
Oct 30 21:52:42.879409 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:42.879388    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-ggdbq"
Oct 30 21:52:43.588279 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:43.588244    4473 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/ai-gateway-controller-6d49f5bf5d-tqzsd"]
Oct 30 21:52:43.592289 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:43.592260    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-6d49f5bf5d-tqzsd"
Oct 30 21:52:43.649389 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:43.649359    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/ai-gateway-controller-6d49f5bf5d-tqzsd"]
Oct 30 21:52:43.671582 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:43.671549    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"ai-gateway-controller-dockercfg-gt9r2\""
Oct 30 21:52:43.672057 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:43.672016    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"ai-gateway-controller-metrics-tls\""
Oct 30 21:52:43.672273 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:43.672223    4473 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"ai-gateway-controller-parameters\""
Oct 30 21:52:43.730471 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:43.730433    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2vm8c\" (UniqueName: \"kubernetes.io/projected/760a7354-8aa1-4e90-86e7-646f3a311267-kube-api-access-2vm8c\") pod \"ai-gateway-controller-6d49f5bf5d-tqzsd\" (UID: \"760a7354-8aa1-4e90-86e7-646f3a311267\") " pod="opendatahub/ai-gateway-controller-6d49f5bf5d-tqzsd"
Oct 30 21:52:43.730654 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:43.730494    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/760a7354-8aa1-4e90-86e7-646f3a311267-metrics-certs\") pod \"ai-gateway-controller-6d49f5bf5d-tqzsd\" (UID: \"760a7354-8aa1-4e90-86e7-646f3a311267\") " pod="opendatahub/ai-gateway-controller-6d49f5bf5d-tqzsd"
Oct 30 21:52:43.831428 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:43.831390    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2vm8c\" (UniqueName: \"kubernetes.io/projected/760a7354-8aa1-4e90-86e7-646f3a311267-kube-api-access-2vm8c\") pod \"ai-gateway-controller-6d49f5bf5d-tqzsd\" (UID: \"760a7354-8aa1-4e90-86e7-646f3a311267\") " pod="opendatahub/ai-gateway-controller-6d49f5bf5d-tqzsd"
Oct 30 21:52:43.831618 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:43.831450    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/760a7354-8aa1-4e90-86e7-646f3a311267-metrics-certs\") pod \"ai-gateway-controller-6d49f5bf5d-tqzsd\" (UID: \"760a7354-8aa1-4e90-86e7-646f3a311267\") " pod="opendatahub/ai-gateway-controller-6d49f5bf5d-tqzsd"
Oct 30 21:52:43.834203 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:43.834172    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/760a7354-8aa1-4e90-86e7-646f3a311267-metrics-certs\") pod \"ai-gateway-controller-6d49f5bf5d-tqzsd\" (UID: \"760a7354-8aa1-4e90-86e7-646f3a311267\") " pod="opendatahub/ai-gateway-controller-6d49f5bf5d-tqzsd"
Oct 30 21:52:43.890701 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:43.890659    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2vm8c\" (UniqueName: \"kubernetes.io/projected/760a7354-8aa1-4e90-86e7-646f3a311267-kube-api-access-2vm8c\") pod \"ai-gateway-controller-6d49f5bf5d-tqzsd\" (UID: \"760a7354-8aa1-4e90-86e7-646f3a311267\") " pod="opendatahub/ai-gateway-controller-6d49f5bf5d-tqzsd"
Oct 30 21:52:43.905424 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:43.905382    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-6d49f5bf5d-tqzsd"
Oct 30 21:52:44.280737 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:44.280713    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/ai-gateway-controller-6d49f5bf5d-tqzsd"]
Oct 30 21:52:44.283625 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:52:44.283594    4473 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod760a7354_8aa1_4e90_86e7_646f3a311267.slice/crio-3c4ededabc9a5702453a4516dc86ce6feacda37ddfdeb1e19f94a1459fc34cb4 WatchSource:0}: Error finding container 3c4ededabc9a5702453a4516dc86ce6feacda37ddfdeb1e19f94a1459fc34cb4: Status 404 returned error can't find the container with id 3c4ededabc9a5702453a4516dc86ce6feacda37ddfdeb1e19f94a1459fc34cb4
Oct 30 21:52:44.284977 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:44.284957    4473 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 21:52:45.026012 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:45.025971    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-6d49f5bf5d-tqzsd" event={"ID":"760a7354-8aa1-4e90-86e7-646f3a311267","Type":"ContainerStarted","Data":"3c4ededabc9a5702453a4516dc86ce6feacda37ddfdeb1e19f94a1459fc34cb4"}
Oct 30 21:52:45.399510 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:45.399475    4473 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-t4dzc"]
Oct 30 21:52:45.404232 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:45.404206    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-t4dzc"
Oct 30 21:52:45.509737 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:45.509697    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-t4dzc"]
Oct 30 21:52:45.550266 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:45.550221    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3172740f-9545-4759-ba06-48beeb526631-istio-token\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-t4dzc\" (UID: \"3172740f-9545-4759-ba06-48beeb526631\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-t4dzc"
Oct 30 21:52:45.550266 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:45.550264    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3172740f-9545-4759-ba06-48beeb526631-credential-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-t4dzc\" (UID: \"3172740f-9545-4759-ba06-48beeb526631\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-t4dzc"
Oct 30 21:52:45.550503 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:45.550343    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3172740f-9545-4759-ba06-48beeb526631-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-t4dzc\" (UID: \"3172740f-9545-4759-ba06-48beeb526631\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-t4dzc"
Oct 30 21:52:45.550503 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:45.550379    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c2l8c\" (UniqueName: \"kubernetes.io/projected/3172740f-9545-4759-ba06-48beeb526631-kube-api-access-c2l8c\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-t4dzc\" (UID: \"3172740f-9545-4759-ba06-48beeb526631\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-t4dzc"
Oct 30 21:52:45.550503 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:45.550493    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3172740f-9545-4759-ba06-48beeb526631-workload-certs\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-t4dzc\" (UID: \"3172740f-9545-4759-ba06-48beeb526631\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-t4dzc"
Oct 30 21:52:45.550605 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:45.550535    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3172740f-9545-4759-ba06-48beeb526631-istio-data\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-t4dzc\" (UID: \"3172740f-9545-4759-ba06-48beeb526631\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-t4dzc"
Oct 30 21:52:45.550605 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:45.550558    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3172740f-9545-4759-ba06-48beeb526631-workload-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-t4dzc\" (UID: \"3172740f-9545-4759-ba06-48beeb526631\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-t4dzc"
Oct 30 21:52:45.550669 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:45.550635    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3172740f-9545-4759-ba06-48beeb526631-istio-envoy\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-t4dzc\" (UID: \"3172740f-9545-4759-ba06-48beeb526631\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-t4dzc"
Oct 30 21:52:45.550669 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:45.550658    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3172740f-9545-4759-ba06-48beeb526631-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-t4dzc\" (UID: \"3172740f-9545-4759-ba06-48beeb526631\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-t4dzc"
Oct 30 21:52:45.651556 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:45.651470    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3172740f-9545-4759-ba06-48beeb526631-workload-certs\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-t4dzc\" (UID: \"3172740f-9545-4759-ba06-48beeb526631\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-t4dzc"
Oct 30 21:52:45.651556 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:45.651520    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3172740f-9545-4759-ba06-48beeb526631-istio-data\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-t4dzc\" (UID: \"3172740f-9545-4759-ba06-48beeb526631\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-t4dzc"
Oct 30 21:52:45.651556 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:45.651542    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3172740f-9545-4759-ba06-48beeb526631-workload-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-t4dzc\" (UID: \"3172740f-9545-4759-ba06-48beeb526631\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-t4dzc"
Oct 30 21:52:45.651841 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:45.651564    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3172740f-9545-4759-ba06-48beeb526631-istio-envoy\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-t4dzc\" (UID: \"3172740f-9545-4759-ba06-48beeb526631\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-t4dzc"
Oct 30 21:52:45.651841 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:45.651695    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3172740f-9545-4759-ba06-48beeb526631-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-t4dzc\" (UID: \"3172740f-9545-4759-ba06-48beeb526631\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-t4dzc"
Oct 30 21:52:45.651841 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:45.651753    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3172740f-9545-4759-ba06-48beeb526631-istio-token\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-t4dzc\" (UID: \"3172740f-9545-4759-ba06-48beeb526631\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-t4dzc"
Oct 30 21:52:45.651841 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:45.651784    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3172740f-9545-4759-ba06-48beeb526631-credential-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-t4dzc\" (UID: \"3172740f-9545-4759-ba06-48beeb526631\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-t4dzc"
Oct 30 21:52:45.651841 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:45.651815    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3172740f-9545-4759-ba06-48beeb526631-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-t4dzc\" (UID: \"3172740f-9545-4759-ba06-48beeb526631\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-t4dzc"
Oct 30 21:52:45.651841 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:45.651839    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c2l8c\" (UniqueName: \"kubernetes.io/projected/3172740f-9545-4759-ba06-48beeb526631-kube-api-access-c2l8c\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-t4dzc\" (UID: \"3172740f-9545-4759-ba06-48beeb526631\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-t4dzc"
Oct 30 21:52:45.652182 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:45.652015    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3172740f-9545-4759-ba06-48beeb526631-istio-data\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-t4dzc\" (UID: \"3172740f-9545-4759-ba06-48beeb526631\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-t4dzc"
Oct 30 21:52:45.652182 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:45.652042    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3172740f-9545-4759-ba06-48beeb526631-workload-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-t4dzc\" (UID: \"3172740f-9545-4759-ba06-48beeb526631\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-t4dzc"
Oct 30 21:52:45.652182 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:45.652070    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3172740f-9545-4759-ba06-48beeb526631-workload-certs\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-t4dzc\" (UID: \"3172740f-9545-4759-ba06-48beeb526631\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-t4dzc"
Oct 30 21:52:45.652334 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:45.652269    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3172740f-9545-4759-ba06-48beeb526631-credential-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-t4dzc\" (UID: \"3172740f-9545-4759-ba06-48beeb526631\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-t4dzc"
Oct 30 21:52:45.652637 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:45.652621    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3172740f-9545-4759-ba06-48beeb526631-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-t4dzc\" (UID: \"3172740f-9545-4759-ba06-48beeb526631\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-t4dzc"
Oct 30 21:52:45.654121 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:45.654098    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3172740f-9545-4759-ba06-48beeb526631-istio-envoy\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-t4dzc\" (UID: \"3172740f-9545-4759-ba06-48beeb526631\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-t4dzc"
Oct 30 21:52:45.654442 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:45.654423    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3172740f-9545-4759-ba06-48beeb526631-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-t4dzc\" (UID: \"3172740f-9545-4759-ba06-48beeb526631\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-t4dzc"
Oct 30 21:52:45.709690 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:45.709659    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3172740f-9545-4759-ba06-48beeb526631-istio-token\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-t4dzc\" (UID: \"3172740f-9545-4759-ba06-48beeb526631\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-t4dzc"
Oct 30 21:52:45.709971 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:45.709949    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c2l8c\" (UniqueName: \"kubernetes.io/projected/3172740f-9545-4759-ba06-48beeb526631-kube-api-access-c2l8c\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-t4dzc\" (UID: \"3172740f-9545-4759-ba06-48beeb526631\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-t4dzc"
Oct 30 21:52:45.718673 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:45.718642    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-t4dzc"
Oct 30 21:52:46.150931 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:46.150891    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-t4dzc"]
Oct 30 21:52:46.187767 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:52:46.187724    4473 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3172740f_9545_4759_ba06_48beeb526631.slice/crio-ba0c5c6ce3f8cc2a44301f0b36b0f54cd4fc19e64d36f91194efd7efcc6fab74 WatchSource:0}: Error finding container ba0c5c6ce3f8cc2a44301f0b36b0f54cd4fc19e64d36f91194efd7efcc6fab74: Status 404 returned error can't find the container with id ba0c5c6ce3f8cc2a44301f0b36b0f54cd4fc19e64d36f91194efd7efcc6fab74
Oct 30 21:52:46.191090 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:46.191046    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:52:46.191205 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:46.191133    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:52:46.191205 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:46.191177    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:52:47.055791 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:47.055750    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-t4dzc" event={"ID":"3172740f-9545-4759-ba06-48beeb526631","Type":"ContainerStarted","Data":"9c90e8240ca667e36a237ce56d20713cfa33931cef6aabbde9155bd226c30fcd"}
Oct 30 21:52:47.056002 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:47.055809    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-t4dzc" event={"ID":"3172740f-9545-4759-ba06-48beeb526631","Type":"ContainerStarted","Data":"ba0c5c6ce3f8cc2a44301f0b36b0f54cd4fc19e64d36f91194efd7efcc6fab74"}
Oct 30 21:52:47.297813 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:47.297754    4473 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-t4dzc" podStartSLOduration=2.297740909 podStartE2EDuration="2.297740909s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:52:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:52:47.246493867 +0000 UTC m=+613.019862355" watchObservedRunningTime="2026-10-30 21:52:47.297740909 +0000 UTC m=+613.071109395"
Oct 30 21:52:47.719083 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:47.719042    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-t4dzc"
Oct 30 21:52:47.720506 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:47.720480    4473 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-t4dzc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" start-of-body=
Oct 30 21:52:47.720621 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:47.720534    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-t4dzc" podUID="3172740f-9545-4759-ba06-48beeb526631" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused"
Oct 30 21:52:48.719736 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:48.719692    4473 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-t4dzc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" start-of-body=
Oct 30 21:52:48.720184 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:48.719756    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-t4dzc" podUID="3172740f-9545-4759-ba06-48beeb526631" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused"
Oct 30 21:52:49.719386 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:49.719347    4473 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-t4dzc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" start-of-body=
Oct 30 21:52:49.719570 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:49.719418    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-t4dzc" podUID="3172740f-9545-4759-ba06-48beeb526631" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused"
Oct 30 21:52:50.324259 ip-10-0-140-18 kubenswrapper[4473]: E1030 21:52:50.324225    4473 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fncvg" containerName="extractor"
Oct 30 21:52:50.719357 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:50.719316    4473 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-t4dzc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" start-of-body=
Oct 30 21:52:50.719547 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:50.719375    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-t4dzc" podUID="3172740f-9545-4759-ba06-48beeb526631" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused"
Oct 30 21:52:51.077053 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:51.076957    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-6d49f5bf5d-tqzsd" event={"ID":"760a7354-8aa1-4e90-86e7-646f3a311267","Type":"ContainerStarted","Data":"248b0981f17515b2a353794ea5e990c0c6de644f91baaa8cbe960695398c2c79"}
Oct 30 21:52:51.077236 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:51.077063    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/ai-gateway-controller-6d49f5bf5d-tqzsd"
Oct 30 21:52:51.205189 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:51.205131    4473 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/ai-gateway-controller-6d49f5bf5d-tqzsd" podStartSLOduration=2.316167133 podStartE2EDuration="8.205114777s" totalImagesPullingTime="5.888947644s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:52:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:52:51.203499648 +0000 UTC m=+616.976868135" watchObservedRunningTime="2026-10-30 21:52:51.205114777 +0000 UTC m=+616.978483263"
Oct 30 21:52:51.719242 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:51.719204    4473 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-t4dzc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" start-of-body=
Oct 30 21:52:51.719624 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:51.719280    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-t4dzc" podUID="3172740f-9545-4759-ba06-48beeb526631" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused"
Oct 30 21:52:52.719356 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:52.719312    4473 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-t4dzc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" start-of-body=
Oct 30 21:52:52.719821 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:52.719385    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-t4dzc" podUID="3172740f-9545-4759-ba06-48beeb526631" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused"
Oct 30 21:52:53.461631 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:53.461595    4473 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7b98bb678c-jg6ld"]
Oct 30 21:52:53.465335 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:53.465314    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7b98bb678c-jg6ld"
Oct 30 21:52:53.523284 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:53.523242    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7b98bb678c-jg6ld"]
Oct 30 21:52:53.543682 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:53.543587    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z4pf9\" (UniqueName: \"kubernetes.io/projected/540be0d8-cd2b-4397-8be5-3da2d9326eee-kube-api-access-z4pf9\") pod \"payload-pre-processing-7b98bb678c-jg6ld\" (UID: \"540be0d8-cd2b-4397-8be5-3da2d9326eee\") " pod="openshift-ingress/payload-pre-processing-7b98bb678c-jg6ld"
Oct 30 21:52:53.543879 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:53.543720    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/540be0d8-cd2b-4397-8be5-3da2d9326eee-plugins-config-volume\") pod \"payload-pre-processing-7b98bb678c-jg6ld\" (UID: \"540be0d8-cd2b-4397-8be5-3da2d9326eee\") " pod="openshift-ingress/payload-pre-processing-7b98bb678c-jg6ld"
Oct 30 21:52:53.570529 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:53.570497    4473 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Oct 30 21:52:53.570769 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:53.570654    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-lrzfw\""
Oct 30 21:52:53.623977 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:53.623938    4473 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-55fbbbf9dc-8vvh2"]
Oct 30 21:52:53.628858 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:53.628830    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-55fbbbf9dc-8vvh2"
Oct 30 21:52:53.645526 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:53.645477    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/540be0d8-cd2b-4397-8be5-3da2d9326eee-plugins-config-volume\") pod \"payload-pre-processing-7b98bb678c-jg6ld\" (UID: \"540be0d8-cd2b-4397-8be5-3da2d9326eee\") " pod="openshift-ingress/payload-pre-processing-7b98bb678c-jg6ld"
Oct 30 21:52:53.645737 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:53.645717    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z4pf9\" (UniqueName: \"kubernetes.io/projected/540be0d8-cd2b-4397-8be5-3da2d9326eee-kube-api-access-z4pf9\") pod \"payload-pre-processing-7b98bb678c-jg6ld\" (UID: \"540be0d8-cd2b-4397-8be5-3da2d9326eee\") " pod="openshift-ingress/payload-pre-processing-7b98bb678c-jg6ld"
Oct 30 21:52:53.646195 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:53.646174    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/540be0d8-cd2b-4397-8be5-3da2d9326eee-plugins-config-volume\") pod \"payload-pre-processing-7b98bb678c-jg6ld\" (UID: \"540be0d8-cd2b-4397-8be5-3da2d9326eee\") " pod="openshift-ingress/payload-pre-processing-7b98bb678c-jg6ld"
Oct 30 21:52:53.673575 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:53.673541    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-55fbbbf9dc-8vvh2"]
Oct 30 21:52:53.704313 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:53.704274    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z4pf9\" (UniqueName: \"kubernetes.io/projected/540be0d8-cd2b-4397-8be5-3da2d9326eee-kube-api-access-z4pf9\") pod \"payload-pre-processing-7b98bb678c-jg6ld\" (UID: \"540be0d8-cd2b-4397-8be5-3da2d9326eee\") " pod="openshift-ingress/payload-pre-processing-7b98bb678c-jg6ld"
Oct 30 21:52:53.719272 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:53.719172    4473 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-t4dzc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" start-of-body=
Oct 30 21:52:53.719272 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:53.719227    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-t4dzc" podUID="3172740f-9545-4759-ba06-48beeb526631" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused"
Oct 30 21:52:53.747441 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:53.747383    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/61fae073-2a6c-422c-9e51-26cdaff5da9f-plugins-config-volume\") pod \"payload-processing-55fbbbf9dc-8vvh2\" (UID: \"61fae073-2a6c-422c-9e51-26cdaff5da9f\") " pod="openshift-ingress/payload-processing-55fbbbf9dc-8vvh2"
Oct 30 21:52:53.747652 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:53.747548    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r84xx\" (UniqueName: \"kubernetes.io/projected/61fae073-2a6c-422c-9e51-26cdaff5da9f-kube-api-access-r84xx\") pod \"payload-processing-55fbbbf9dc-8vvh2\" (UID: \"61fae073-2a6c-422c-9e51-26cdaff5da9f\") " pod="openshift-ingress/payload-processing-55fbbbf9dc-8vvh2"
Oct 30 21:52:53.779938 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:53.779874    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7b98bb678c-jg6ld"
Oct 30 21:52:53.849233 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:53.848978    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r84xx\" (UniqueName: \"kubernetes.io/projected/61fae073-2a6c-422c-9e51-26cdaff5da9f-kube-api-access-r84xx\") pod \"payload-processing-55fbbbf9dc-8vvh2\" (UID: \"61fae073-2a6c-422c-9e51-26cdaff5da9f\") " pod="openshift-ingress/payload-processing-55fbbbf9dc-8vvh2"
Oct 30 21:52:53.849233 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:53.849117    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/61fae073-2a6c-422c-9e51-26cdaff5da9f-plugins-config-volume\") pod \"payload-processing-55fbbbf9dc-8vvh2\" (UID: \"61fae073-2a6c-422c-9e51-26cdaff5da9f\") " pod="openshift-ingress/payload-processing-55fbbbf9dc-8vvh2"
Oct 30 21:52:53.850136 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:53.850106    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/61fae073-2a6c-422c-9e51-26cdaff5da9f-plugins-config-volume\") pod \"payload-processing-55fbbbf9dc-8vvh2\" (UID: \"61fae073-2a6c-422c-9e51-26cdaff5da9f\") " pod="openshift-ingress/payload-processing-55fbbbf9dc-8vvh2"
Oct 30 21:52:53.909601 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:53.909557    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r84xx\" (UniqueName: \"kubernetes.io/projected/61fae073-2a6c-422c-9e51-26cdaff5da9f-kube-api-access-r84xx\") pod \"payload-processing-55fbbbf9dc-8vvh2\" (UID: \"61fae073-2a6c-422c-9e51-26cdaff5da9f\") " pod="openshift-ingress/payload-processing-55fbbbf9dc-8vvh2"
Oct 30 21:52:53.944283 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:53.944242    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-55fbbbf9dc-8vvh2"
Oct 30 21:52:54.177236 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:54.177212    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7b98bb678c-jg6ld"]
Oct 30 21:52:54.180132 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:52:54.180100    4473 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod540be0d8_cd2b_4397_8be5_3da2d9326eee.slice/crio-5f87a63bbe0e322da467e420fc155e9dbe17e569e419c5f12951b4479bb3d111 WatchSource:0}: Error finding container 5f87a63bbe0e322da467e420fc155e9dbe17e569e419c5f12951b4479bb3d111: Status 404 returned error can't find the container with id 5f87a63bbe0e322da467e420fc155e9dbe17e569e419c5f12951b4479bb3d111
Oct 30 21:52:54.335891 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:54.335864    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-55fbbbf9dc-8vvh2"]
Oct 30 21:52:54.338083 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:52:54.338049    4473 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod61fae073_2a6c_422c_9e51_26cdaff5da9f.slice/crio-a9dff247f83953b3ce262a648d4bfb8202e4fb0ba198813d7355abe96c17c701 WatchSource:0}: Error finding container a9dff247f83953b3ce262a648d4bfb8202e4fb0ba198813d7355abe96c17c701: Status 404 returned error can't find the container with id a9dff247f83953b3ce262a648d4bfb8202e4fb0ba198813d7355abe96c17c701
Oct 30 21:52:55.102963 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:55.102907    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-55fbbbf9dc-8vvh2" event={"ID":"61fae073-2a6c-422c-9e51-26cdaff5da9f","Type":"ContainerStarted","Data":"a9dff247f83953b3ce262a648d4bfb8202e4fb0ba198813d7355abe96c17c701"}
Oct 30 21:52:55.104383 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:55.104338    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7b98bb678c-jg6ld" event={"ID":"540be0d8-cd2b-4397-8be5-3da2d9326eee","Type":"ContainerStarted","Data":"5f87a63bbe0e322da467e420fc155e9dbe17e569e419c5f12951b4479bb3d111"}
Oct 30 21:52:55.720184 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:55.720144    4473 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-t4dzc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:52:55.720405 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:55.720217    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-t4dzc" podUID="3172740f-9545-4759-ba06-48beeb526631" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:52:56.763180 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:56.763128    4473 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-t4dzc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:52:56.763706 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:56.763219    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-t4dzc" podUID="3172740f-9545-4759-ba06-48beeb526631" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:52:57.466475 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:57.466443    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-t4dzc"
Oct 30 21:52:57.467266 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:57.467205    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-t4dzc"
Oct 30 21:52:57.468232 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:57.468211    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-t4dzc"
Oct 30 21:52:58.123116 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:58.123071    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-55fbbbf9dc-8vvh2" event={"ID":"61fae073-2a6c-422c-9e51-26cdaff5da9f","Type":"ContainerStarted","Data":"a4b0a40fb954530680a6249e2e3cfe005ac03b2359a68244e5cda87377acc649"}
Oct 30 21:52:58.123624 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:58.123191    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-55fbbbf9dc-8vvh2"
Oct 30 21:52:58.125460 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:58.125423    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7b98bb678c-jg6ld" event={"ID":"540be0d8-cd2b-4397-8be5-3da2d9326eee","Type":"ContainerStarted","Data":"3fe4de028de8d1d0db47d6aebe5b5d1a23472558f7c6a9c306d7a6a9c13a2f56"}
Oct 30 21:52:58.125598 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:58.125532    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7b98bb678c-jg6ld"
Oct 30 21:52:58.242525 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:52:58.242410    4473 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-55fbbbf9dc-8vvh2" podStartSLOduration=1.6712491090000001 podStartE2EDuration="5.242395265s" totalImagesPullingTime="3.571146156s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:52:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:52:58.24067594 +0000 UTC m=+624.014044429" watchObservedRunningTime="2026-10-30 21:52:58.242395265 +0000 UTC m=+624.015763789"
Oct 30 21:53:02.084621 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:02.084588    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/ai-gateway-controller-6d49f5bf5d-tqzsd"
Oct 30 21:53:02.201671 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:02.201623    4473 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7b98bb678c-jg6ld" podStartSLOduration=5.481282444 podStartE2EDuration="9.201609436s" totalImagesPullingTime="3.720326992s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:52:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:52:58.368627195 +0000 UTC m=+624.141995683" watchObservedRunningTime="2026-10-30 21:53:02.201609436 +0000 UTC m=+627.974977923"
Oct 30 21:53:09.133150 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:09.133122    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-55fbbbf9dc-8vvh2"
Oct 30 21:53:09.133545 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:09.133187    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7b98bb678c-jg6ld"
Oct 30 21:53:17.648134 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:17.648098    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/ai-gateway-controller-6d49f5bf5d-tqzsd"]
Oct 30 21:53:17.648634 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:17.648391    4473 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/ai-gateway-controller-6d49f5bf5d-tqzsd" podUID="760a7354-8aa1-4e90-86e7-646f3a311267" containerName="manager" containerID="cri-o://248b0981f17515b2a353794ea5e990c0c6de644f91baaa8cbe960695398c2c79" gracePeriod=30
Oct 30 21:53:17.898501 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:17.898433    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-6d49f5bf5d-tqzsd"
Oct 30 21:53:18.005086 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:18.005046    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/760a7354-8aa1-4e90-86e7-646f3a311267-kube-api-access-2vm8c\" (UniqueName: \"kubernetes.io/projected/760a7354-8aa1-4e90-86e7-646f3a311267-kube-api-access-2vm8c\") pod \"760a7354-8aa1-4e90-86e7-646f3a311267\" (UID: \"760a7354-8aa1-4e90-86e7-646f3a311267\") "
Oct 30 21:53:18.005285 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:18.005112    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/760a7354-8aa1-4e90-86e7-646f3a311267-metrics-certs\" (UniqueName: \"kubernetes.io/secret/760a7354-8aa1-4e90-86e7-646f3a311267-metrics-certs\") pod \"760a7354-8aa1-4e90-86e7-646f3a311267\" (UID: \"760a7354-8aa1-4e90-86e7-646f3a311267\") "
Oct 30 21:53:18.007347 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:18.007315    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/760a7354-8aa1-4e90-86e7-646f3a311267-kube-api-access-2vm8c" pod "760a7354-8aa1-4e90-86e7-646f3a311267" (UID: "760a7354-8aa1-4e90-86e7-646f3a311267"). InnerVolumeSpecName "kube-api-access-2vm8c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:53:18.007458 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:18.007415    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/760a7354-8aa1-4e90-86e7-646f3a311267-metrics-certs" pod "760a7354-8aa1-4e90-86e7-646f3a311267" (UID: "760a7354-8aa1-4e90-86e7-646f3a311267"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:53:18.106473 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:18.106430    4473 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2vm8c\" (UniqueName: \"kubernetes.io/projected/760a7354-8aa1-4e90-86e7-646f3a311267-kube-api-access-2vm8c\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:53:18.106473 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:18.106474    4473 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/760a7354-8aa1-4e90-86e7-646f3a311267-metrics-certs\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:53:18.223822 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:18.223738    4473 generic.go:356] "Generic (PLEG): container finished" podID="760a7354-8aa1-4e90-86e7-646f3a311267" containerID="248b0981f17515b2a353794ea5e990c0c6de644f91baaa8cbe960695398c2c79" exitCode=0
Oct 30 21:53:18.223822 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:18.223787    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-6d49f5bf5d-tqzsd" event={"ID":"760a7354-8aa1-4e90-86e7-646f3a311267","Type":"ContainerDied","Data":"248b0981f17515b2a353794ea5e990c0c6de644f91baaa8cbe960695398c2c79"}
Oct 30 21:53:18.223822 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:18.223790    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-6d49f5bf5d-tqzsd"
Oct 30 21:53:18.224094 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:18.223829    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-6d49f5bf5d-tqzsd" event={"ID":"760a7354-8aa1-4e90-86e7-646f3a311267","Type":"ContainerDied","Data":"3c4ededabc9a5702453a4516dc86ce6feacda37ddfdeb1e19f94a1459fc34cb4"}
Oct 30 21:53:18.224094 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:18.223852    4473 scope.go:122] "RemoveContainer" containerID="248b0981f17515b2a353794ea5e990c0c6de644f91baaa8cbe960695398c2c79"
Oct 30 21:53:18.233505 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:18.233486    4473 scope.go:122] "RemoveContainer" containerID="248b0981f17515b2a353794ea5e990c0c6de644f91baaa8cbe960695398c2c79"
Oct 30 21:53:18.233838 ip-10-0-140-18 kubenswrapper[4473]: E1030 21:53:18.233822    4473 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"248b0981f17515b2a353794ea5e990c0c6de644f91baaa8cbe960695398c2c79\": container with ID starting with 248b0981f17515b2a353794ea5e990c0c6de644f91baaa8cbe960695398c2c79 not found: ID does not exist" containerID="248b0981f17515b2a353794ea5e990c0c6de644f91baaa8cbe960695398c2c79"
Oct 30 21:53:18.233888 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:18.233852    4473 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"248b0981f17515b2a353794ea5e990c0c6de644f91baaa8cbe960695398c2c79"} err="failed to get container status \"248b0981f17515b2a353794ea5e990c0c6de644f91baaa8cbe960695398c2c79\": rpc error: code = NotFound desc = could not find container \"248b0981f17515b2a353794ea5e990c0c6de644f91baaa8cbe960695398c2c79\": container with ID starting with 248b0981f17515b2a353794ea5e990c0c6de644f91baaa8cbe960695398c2c79 not found: ID does not exist"
Oct 30 21:53:18.390629 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:18.390585    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/ai-gateway-controller-6d49f5bf5d-tqzsd"]
Oct 30 21:53:18.441098 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:18.441054    4473 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/ai-gateway-controller-6d49f5bf5d-tqzsd"]
Oct 30 21:53:20.324449 ip-10-0-140-18 kubenswrapper[4473]: E1030 21:53:20.324408    4473 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fncvg" containerName="extractor"
Oct 30 21:53:20.355622 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:20.355592    4473 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="760a7354-8aa1-4e90-86e7-646f3a311267" path="/var/lib/kubelet/pods/760a7354-8aa1-4e90-86e7-646f3a311267/volumes"
Oct 30 21:53:49.419757 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:49.419677    4473 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-qng2w"]
Oct 30 21:53:49.420425 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:49.420404    4473 cpu_manager.go:414] "containerMap: removing container" podUID="760a7354-8aa1-4e90-86e7-646f3a311267" containerName="manager"
Oct 30 21:53:49.420469 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:49.420429    4473 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="760a7354-8aa1-4e90-86e7-646f3a311267" containerName="manager"
Oct 30 21:53:49.420540 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:49.420527    4473 memory_manager.go:365] "RemoveStaleState removing state" podUID="760a7354-8aa1-4e90-86e7-646f3a311267" containerName="manager"
Oct 30 21:53:49.423374 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:49.423353    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-qng2w"
Oct 30 21:53:49.486487 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:49.486453    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w2-0c99b1-openshift-default-dockercfg-r8hwd\""
Oct 30 21:53:49.507704 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:49.507666    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6560c1bc-b507-4a92-8586-23e0fcc5362a-istio-token\") pod \"e2e-worker-w2-0c99b1-openshift-default-967855977-qng2w\" (UID: \"6560c1bc-b507-4a92-8586-23e0fcc5362a\") " pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-qng2w"
Oct 30 21:53:49.507855 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:49.507713    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6560c1bc-b507-4a92-8586-23e0fcc5362a-istiod-ca-cert\") pod \"e2e-worker-w2-0c99b1-openshift-default-967855977-qng2w\" (UID: \"6560c1bc-b507-4a92-8586-23e0fcc5362a\") " pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-qng2w"
Oct 30 21:53:49.507855 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:49.507739    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6560c1bc-b507-4a92-8586-23e0fcc5362a-istio-podinfo\") pod \"e2e-worker-w2-0c99b1-openshift-default-967855977-qng2w\" (UID: \"6560c1bc-b507-4a92-8586-23e0fcc5362a\") " pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-qng2w"
Oct 30 21:53:49.507996 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:49.507843    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6560c1bc-b507-4a92-8586-23e0fcc5362a-credential-socket\") pod \"e2e-worker-w2-0c99b1-openshift-default-967855977-qng2w\" (UID: \"6560c1bc-b507-4a92-8586-23e0fcc5362a\") " pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-qng2w"
Oct 30 21:53:49.507996 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:49.507883    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6560c1bc-b507-4a92-8586-23e0fcc5362a-istio-envoy\") pod \"e2e-worker-w2-0c99b1-openshift-default-967855977-qng2w\" (UID: \"6560c1bc-b507-4a92-8586-23e0fcc5362a\") " pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-qng2w"
Oct 30 21:53:49.507996 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:49.507913    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6560c1bc-b507-4a92-8586-23e0fcc5362a-workload-certs\") pod \"e2e-worker-w2-0c99b1-openshift-default-967855977-qng2w\" (UID: \"6560c1bc-b507-4a92-8586-23e0fcc5362a\") " pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-qng2w"
Oct 30 21:53:49.508107 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:49.508033    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qpsj6\" (UniqueName: \"kubernetes.io/projected/6560c1bc-b507-4a92-8586-23e0fcc5362a-kube-api-access-qpsj6\") pod \"e2e-worker-w2-0c99b1-openshift-default-967855977-qng2w\" (UID: \"6560c1bc-b507-4a92-8586-23e0fcc5362a\") " pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-qng2w"
Oct 30 21:53:49.508107 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:49.508065    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6560c1bc-b507-4a92-8586-23e0fcc5362a-istio-data\") pod \"e2e-worker-w2-0c99b1-openshift-default-967855977-qng2w\" (UID: \"6560c1bc-b507-4a92-8586-23e0fcc5362a\") " pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-qng2w"
Oct 30 21:53:49.508175 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:49.508118    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6560c1bc-b507-4a92-8586-23e0fcc5362a-workload-socket\") pod \"e2e-worker-w2-0c99b1-openshift-default-967855977-qng2w\" (UID: \"6560c1bc-b507-4a92-8586-23e0fcc5362a\") " pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-qng2w"
Oct 30 21:53:49.534962 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:49.534886    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-qng2w"]
Oct 30 21:53:49.608782 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:49.608746    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6560c1bc-b507-4a92-8586-23e0fcc5362a-istiod-ca-cert\") pod \"e2e-worker-w2-0c99b1-openshift-default-967855977-qng2w\" (UID: \"6560c1bc-b507-4a92-8586-23e0fcc5362a\") " pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-qng2w"
Oct 30 21:53:49.608782 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:49.608784    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6560c1bc-b507-4a92-8586-23e0fcc5362a-istio-podinfo\") pod \"e2e-worker-w2-0c99b1-openshift-default-967855977-qng2w\" (UID: \"6560c1bc-b507-4a92-8586-23e0fcc5362a\") " pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-qng2w"
Oct 30 21:53:49.609055 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:49.608813    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6560c1bc-b507-4a92-8586-23e0fcc5362a-credential-socket\") pod \"e2e-worker-w2-0c99b1-openshift-default-967855977-qng2w\" (UID: \"6560c1bc-b507-4a92-8586-23e0fcc5362a\") " pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-qng2w"
Oct 30 21:53:49.609055 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:49.608838    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6560c1bc-b507-4a92-8586-23e0fcc5362a-istio-envoy\") pod \"e2e-worker-w2-0c99b1-openshift-default-967855977-qng2w\" (UID: \"6560c1bc-b507-4a92-8586-23e0fcc5362a\") " pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-qng2w"
Oct 30 21:53:49.609055 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:49.608861    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6560c1bc-b507-4a92-8586-23e0fcc5362a-workload-certs\") pod \"e2e-worker-w2-0c99b1-openshift-default-967855977-qng2w\" (UID: \"6560c1bc-b507-4a92-8586-23e0fcc5362a\") " pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-qng2w"
Oct 30 21:53:49.609055 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:49.608897    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qpsj6\" (UniqueName: \"kubernetes.io/projected/6560c1bc-b507-4a92-8586-23e0fcc5362a-kube-api-access-qpsj6\") pod \"e2e-worker-w2-0c99b1-openshift-default-967855977-qng2w\" (UID: \"6560c1bc-b507-4a92-8586-23e0fcc5362a\") " pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-qng2w"
Oct 30 21:53:49.609055 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:49.608945    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6560c1bc-b507-4a92-8586-23e0fcc5362a-istio-data\") pod \"e2e-worker-w2-0c99b1-openshift-default-967855977-qng2w\" (UID: \"6560c1bc-b507-4a92-8586-23e0fcc5362a\") " pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-qng2w"
Oct 30 21:53:49.609055 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:49.609004    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6560c1bc-b507-4a92-8586-23e0fcc5362a-workload-socket\") pod \"e2e-worker-w2-0c99b1-openshift-default-967855977-qng2w\" (UID: \"6560c1bc-b507-4a92-8586-23e0fcc5362a\") " pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-qng2w"
Oct 30 21:53:49.609313 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:49.609060    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6560c1bc-b507-4a92-8586-23e0fcc5362a-istio-token\") pod \"e2e-worker-w2-0c99b1-openshift-default-967855977-qng2w\" (UID: \"6560c1bc-b507-4a92-8586-23e0fcc5362a\") " pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-qng2w"
Oct 30 21:53:49.609352 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:49.609324    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6560c1bc-b507-4a92-8586-23e0fcc5362a-workload-certs\") pod \"e2e-worker-w2-0c99b1-openshift-default-967855977-qng2w\" (UID: \"6560c1bc-b507-4a92-8586-23e0fcc5362a\") " pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-qng2w"
Oct 30 21:53:49.609388 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:49.609377    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6560c1bc-b507-4a92-8586-23e0fcc5362a-credential-socket\") pod \"e2e-worker-w2-0c99b1-openshift-default-967855977-qng2w\" (UID: \"6560c1bc-b507-4a92-8586-23e0fcc5362a\") " pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-qng2w"
Oct 30 21:53:49.609556 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:49.609537    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6560c1bc-b507-4a92-8586-23e0fcc5362a-istiod-ca-cert\") pod \"e2e-worker-w2-0c99b1-openshift-default-967855977-qng2w\" (UID: \"6560c1bc-b507-4a92-8586-23e0fcc5362a\") " pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-qng2w"
Oct 30 21:53:49.609642 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:49.609620    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6560c1bc-b507-4a92-8586-23e0fcc5362a-workload-socket\") pod \"e2e-worker-w2-0c99b1-openshift-default-967855977-qng2w\" (UID: \"6560c1bc-b507-4a92-8586-23e0fcc5362a\") " pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-qng2w"
Oct 30 21:53:49.609690 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:49.609623    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6560c1bc-b507-4a92-8586-23e0fcc5362a-istio-data\") pod \"e2e-worker-w2-0c99b1-openshift-default-967855977-qng2w\" (UID: \"6560c1bc-b507-4a92-8586-23e0fcc5362a\") " pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-qng2w"
Oct 30 21:53:49.611481 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:49.611459    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6560c1bc-b507-4a92-8586-23e0fcc5362a-istio-envoy\") pod \"e2e-worker-w2-0c99b1-openshift-default-967855977-qng2w\" (UID: \"6560c1bc-b507-4a92-8586-23e0fcc5362a\") " pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-qng2w"
Oct 30 21:53:49.612022 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:49.612005    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6560c1bc-b507-4a92-8586-23e0fcc5362a-istio-podinfo\") pod \"e2e-worker-w2-0c99b1-openshift-default-967855977-qng2w\" (UID: \"6560c1bc-b507-4a92-8586-23e0fcc5362a\") " pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-qng2w"
Oct 30 21:53:49.669536 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:49.669498    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6560c1bc-b507-4a92-8586-23e0fcc5362a-istio-token\") pod \"e2e-worker-w2-0c99b1-openshift-default-967855977-qng2w\" (UID: \"6560c1bc-b507-4a92-8586-23e0fcc5362a\") " pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-qng2w"
Oct 30 21:53:49.670228 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:49.670180    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qpsj6\" (UniqueName: \"kubernetes.io/projected/6560c1bc-b507-4a92-8586-23e0fcc5362a-kube-api-access-qpsj6\") pod \"e2e-worker-w2-0c99b1-openshift-default-967855977-qng2w\" (UID: \"6560c1bc-b507-4a92-8586-23e0fcc5362a\") " pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-qng2w"
Oct 30 21:53:49.738004 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:49.737969    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-qng2w"
Oct 30 21:53:50.051441 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:50.051354    4473 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-xct57"]
Oct 30 21:53:50.057438 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:50.057408    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-xct57"
Oct 30 21:53:50.113597 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:50.113565    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w8pmk\" (UniqueName: \"kubernetes.io/projected/893a5438-3b54-4d77-8675-00260980c0f6-kube-api-access-w8pmk\") pod \"e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-xct57\" (UID: \"893a5438-3b54-4d77-8675-00260980c0f6\") " pod="openshift-ingress/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-xct57"
Oct 30 21:53:50.113597 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:50.113609    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/893a5438-3b54-4d77-8675-00260980c0f6-istio-token\") pod \"e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-xct57\" (UID: \"893a5438-3b54-4d77-8675-00260980c0f6\") " pod="openshift-ingress/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-xct57"
Oct 30 21:53:50.113811 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:50.113629    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/893a5438-3b54-4d77-8675-00260980c0f6-istio-podinfo\") pod \"e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-xct57\" (UID: \"893a5438-3b54-4d77-8675-00260980c0f6\") " pod="openshift-ingress/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-xct57"
Oct 30 21:53:50.113811 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:50.113751    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/893a5438-3b54-4d77-8675-00260980c0f6-istiod-ca-cert\") pod \"e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-xct57\" (UID: \"893a5438-3b54-4d77-8675-00260980c0f6\") " pod="openshift-ingress/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-xct57"
Oct 30 21:53:50.113890 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:50.113819    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/893a5438-3b54-4d77-8675-00260980c0f6-istio-data\") pod \"e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-xct57\" (UID: \"893a5438-3b54-4d77-8675-00260980c0f6\") " pod="openshift-ingress/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-xct57"
Oct 30 21:53:50.113968 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:50.113938    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/893a5438-3b54-4d77-8675-00260980c0f6-credential-socket\") pod \"e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-xct57\" (UID: \"893a5438-3b54-4d77-8675-00260980c0f6\") " pod="openshift-ingress/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-xct57"
Oct 30 21:53:50.114016 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:50.113984    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/893a5438-3b54-4d77-8675-00260980c0f6-workload-socket\") pod \"e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-xct57\" (UID: \"893a5438-3b54-4d77-8675-00260980c0f6\") " pod="openshift-ingress/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-xct57"
Oct 30 21:53:50.114016 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:50.114006    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/893a5438-3b54-4d77-8675-00260980c0f6-workload-certs\") pod \"e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-xct57\" (UID: \"893a5438-3b54-4d77-8675-00260980c0f6\") " pod="openshift-ingress/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-xct57"
Oct 30 21:53:50.114114 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:50.114094    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/893a5438-3b54-4d77-8675-00260980c0f6-istio-envoy\") pod \"e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-xct57\" (UID: \"893a5438-3b54-4d77-8675-00260980c0f6\") " pod="openshift-ingress/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-xct57"
Oct 30 21:53:50.114943 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:50.114886    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w5-9f8aea-openshift-default-dockercfg-64lqz\""
Oct 30 21:53:50.118141 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:50.118117    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-xct57"]
Oct 30 21:53:50.214626 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:50.214598    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-qng2w"]
Oct 30 21:53:50.215166 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:50.215123    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/893a5438-3b54-4d77-8675-00260980c0f6-istio-data\") pod \"e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-xct57\" (UID: \"893a5438-3b54-4d77-8675-00260980c0f6\") " pod="openshift-ingress/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-xct57"
Oct 30 21:53:50.215282 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:50.215176    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/893a5438-3b54-4d77-8675-00260980c0f6-credential-socket\") pod \"e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-xct57\" (UID: \"893a5438-3b54-4d77-8675-00260980c0f6\") " pod="openshift-ingress/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-xct57"
Oct 30 21:53:50.215282 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:50.215212    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/893a5438-3b54-4d77-8675-00260980c0f6-workload-socket\") pod \"e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-xct57\" (UID: \"893a5438-3b54-4d77-8675-00260980c0f6\") " pod="openshift-ingress/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-xct57"
Oct 30 21:53:50.215282 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:50.215239    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/893a5438-3b54-4d77-8675-00260980c0f6-workload-certs\") pod \"e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-xct57\" (UID: \"893a5438-3b54-4d77-8675-00260980c0f6\") " pod="openshift-ingress/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-xct57"
Oct 30 21:53:50.215282 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:50.215271    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/893a5438-3b54-4d77-8675-00260980c0f6-istio-envoy\") pod \"e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-xct57\" (UID: \"893a5438-3b54-4d77-8675-00260980c0f6\") " pod="openshift-ingress/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-xct57"
Oct 30 21:53:50.215432 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:50.215351    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w8pmk\" (UniqueName: \"kubernetes.io/projected/893a5438-3b54-4d77-8675-00260980c0f6-kube-api-access-w8pmk\") pod \"e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-xct57\" (UID: \"893a5438-3b54-4d77-8675-00260980c0f6\") " pod="openshift-ingress/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-xct57"
Oct 30 21:53:50.215432 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:50.215395    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/893a5438-3b54-4d77-8675-00260980c0f6-istio-token\") pod \"e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-xct57\" (UID: \"893a5438-3b54-4d77-8675-00260980c0f6\") " pod="openshift-ingress/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-xct57"
Oct 30 21:53:50.215432 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:50.215426    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/893a5438-3b54-4d77-8675-00260980c0f6-istio-podinfo\") pod \"e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-xct57\" (UID: \"893a5438-3b54-4d77-8675-00260980c0f6\") " pod="openshift-ingress/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-xct57"
Oct 30 21:53:50.215531 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:50.215469    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/893a5438-3b54-4d77-8675-00260980c0f6-istiod-ca-cert\") pod \"e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-xct57\" (UID: \"893a5438-3b54-4d77-8675-00260980c0f6\") " pod="openshift-ingress/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-xct57"
Oct 30 21:53:50.215688 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:50.215668    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/893a5438-3b54-4d77-8675-00260980c0f6-istio-data\") pod \"e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-xct57\" (UID: \"893a5438-3b54-4d77-8675-00260980c0f6\") " pod="openshift-ingress/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-xct57"
Oct 30 21:53:50.216219 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:50.216193    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/893a5438-3b54-4d77-8675-00260980c0f6-istiod-ca-cert\") pod \"e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-xct57\" (UID: \"893a5438-3b54-4d77-8675-00260980c0f6\") " pod="openshift-ingress/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-xct57"
Oct 30 21:53:50.216432 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:50.216413    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/893a5438-3b54-4d77-8675-00260980c0f6-credential-socket\") pod \"e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-xct57\" (UID: \"893a5438-3b54-4d77-8675-00260980c0f6\") " pod="openshift-ingress/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-xct57"
Oct 30 21:53:50.216624 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:50.216582    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/893a5438-3b54-4d77-8675-00260980c0f6-workload-certs\") pod \"e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-xct57\" (UID: \"893a5438-3b54-4d77-8675-00260980c0f6\") " pod="openshift-ingress/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-xct57"
Oct 30 21:53:50.216735 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:50.216709    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/893a5438-3b54-4d77-8675-00260980c0f6-workload-socket\") pod \"e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-xct57\" (UID: \"893a5438-3b54-4d77-8675-00260980c0f6\") " pod="openshift-ingress/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-xct57"
Oct 30 21:53:50.217094 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:53:50.217067    4473 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6560c1bc_b507_4a92_8586_23e0fcc5362a.slice/crio-443f61aee6536dfd73b6485479a5e29f8fa239efcb32f4fac72c9c8086b5ed28 WatchSource:0}: Error finding container 443f61aee6536dfd73b6485479a5e29f8fa239efcb32f4fac72c9c8086b5ed28: Status 404 returned error can't find the container with id 443f61aee6536dfd73b6485479a5e29f8fa239efcb32f4fac72c9c8086b5ed28
Oct 30 21:53:50.220219 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:50.220196    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/893a5438-3b54-4d77-8675-00260980c0f6-istio-envoy\") pod \"e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-xct57\" (UID: \"893a5438-3b54-4d77-8675-00260980c0f6\") " pod="openshift-ingress/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-xct57"
Oct 30 21:53:50.220536 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:50.220512    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/893a5438-3b54-4d77-8675-00260980c0f6-istio-podinfo\") pod \"e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-xct57\" (UID: \"893a5438-3b54-4d77-8675-00260980c0f6\") " pod="openshift-ingress/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-xct57"
Oct 30 21:53:50.222243 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:50.222206    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:53:50.222344 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:50.222300    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:53:50.222407 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:50.222345    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:53:50.274418 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:50.274390    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/893a5438-3b54-4d77-8675-00260980c0f6-istio-token\") pod \"e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-xct57\" (UID: \"893a5438-3b54-4d77-8675-00260980c0f6\") " pod="openshift-ingress/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-xct57"
Oct 30 21:53:50.274577 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:50.274532    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w8pmk\" (UniqueName: \"kubernetes.io/projected/893a5438-3b54-4d77-8675-00260980c0f6-kube-api-access-w8pmk\") pod \"e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-xct57\" (UID: \"893a5438-3b54-4d77-8675-00260980c0f6\") " pod="openshift-ingress/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-xct57"
Oct 30 21:53:50.324523 ip-10-0-140-18 kubenswrapper[4473]: E1030 21:53:50.324434    4473 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fncvg" containerName="extractor"
Oct 30 21:53:50.371192 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:50.371158    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-xct57"
Oct 30 21:53:50.379486 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:50.379446    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-qng2w" event={"ID":"6560c1bc-b507-4a92-8586-23e0fcc5362a","Type":"ContainerStarted","Data":"66d5162ae5be3f6186045d73c5997ea750a278198e32e17fc0b5624548b96834"}
Oct 30 21:53:50.379669 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:50.379519    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-qng2w" event={"ID":"6560c1bc-b507-4a92-8586-23e0fcc5362a","Type":"ContainerStarted","Data":"443f61aee6536dfd73b6485479a5e29f8fa239efcb32f4fac72c9c8086b5ed28"}
Oct 30 21:53:50.514996 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:50.514908    4473 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-qng2w" podStartSLOduration=1.5148872 podStartE2EDuration="1.5148872s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:53:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:53:50.512686732 +0000 UTC m=+676.286055220" watchObservedRunningTime="2026-10-30 21:53:50.5148872 +0000 UTC m=+676.288255688"
Oct 30 21:53:50.710818 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:50.710778    4473 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-bs4lr"]
Oct 30 21:53:50.722201 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:50.722168    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-bs4lr"
Oct 30 21:53:50.744286 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:50.744237    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-qng2w"
Oct 30 21:53:50.777047 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:50.777019    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-bs4lr"]
Oct 30 21:53:50.780285 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:50.780256    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w1-d516c4-openshift-default-dockercfg-mtv65\""
Oct 30 21:53:50.812869 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:50.812845    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-xct57"]
Oct 30 21:53:50.814482 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:53:50.814452    4473 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod893a5438_3b54_4d77_8675_00260980c0f6.slice/crio-f9e49bacba3d0a169bae3b97fd3bdafefb603f38a3fa86113a62ef474dadf00b WatchSource:0}: Error finding container f9e49bacba3d0a169bae3b97fd3bdafefb603f38a3fa86113a62ef474dadf00b: Status 404 returned error can't find the container with id f9e49bacba3d0a169bae3b97fd3bdafefb603f38a3fa86113a62ef474dadf00b
Oct 30 21:53:50.816959 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:50.816913    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:53:50.817059 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:50.816990    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:53:50.817059 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:50.817019    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:53:50.823214 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:50.823188    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/335708bc-083d-4ce1-9158-cc61dcac93da-workload-certs\") pod \"e2e-worker-w1-d516c4-openshift-default-6bd584745-bs4lr\" (UID: \"335708bc-083d-4ce1-9158-cc61dcac93da\") " pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-bs4lr"
Oct 30 21:53:50.823332 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:50.823224    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n865n\" (UniqueName: \"kubernetes.io/projected/335708bc-083d-4ce1-9158-cc61dcac93da-kube-api-access-n865n\") pod \"e2e-worker-w1-d516c4-openshift-default-6bd584745-bs4lr\" (UID: \"335708bc-083d-4ce1-9158-cc61dcac93da\") " pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-bs4lr"
Oct 30 21:53:50.823332 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:50.823256    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/335708bc-083d-4ce1-9158-cc61dcac93da-istio-data\") pod \"e2e-worker-w1-d516c4-openshift-default-6bd584745-bs4lr\" (UID: \"335708bc-083d-4ce1-9158-cc61dcac93da\") " pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-bs4lr"
Oct 30 21:53:50.823621 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:50.823515    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/335708bc-083d-4ce1-9158-cc61dcac93da-istio-podinfo\") pod \"e2e-worker-w1-d516c4-openshift-default-6bd584745-bs4lr\" (UID: \"335708bc-083d-4ce1-9158-cc61dcac93da\") " pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-bs4lr"
Oct 30 21:53:50.823621 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:50.823543    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/335708bc-083d-4ce1-9158-cc61dcac93da-credential-socket\") pod \"e2e-worker-w1-d516c4-openshift-default-6bd584745-bs4lr\" (UID: \"335708bc-083d-4ce1-9158-cc61dcac93da\") " pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-bs4lr"
Oct 30 21:53:50.823735 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:50.823656    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/335708bc-083d-4ce1-9158-cc61dcac93da-istio-token\") pod \"e2e-worker-w1-d516c4-openshift-default-6bd584745-bs4lr\" (UID: \"335708bc-083d-4ce1-9158-cc61dcac93da\") " pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-bs4lr"
Oct 30 21:53:50.823735 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:50.823707    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/335708bc-083d-4ce1-9158-cc61dcac93da-istio-envoy\") pod \"e2e-worker-w1-d516c4-openshift-default-6bd584745-bs4lr\" (UID: \"335708bc-083d-4ce1-9158-cc61dcac93da\") " pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-bs4lr"
Oct 30 21:53:50.823804 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:50.823766    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/335708bc-083d-4ce1-9158-cc61dcac93da-workload-socket\") pod \"e2e-worker-w1-d516c4-openshift-default-6bd584745-bs4lr\" (UID: \"335708bc-083d-4ce1-9158-cc61dcac93da\") " pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-bs4lr"
Oct 30 21:53:50.823804 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:50.823792    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/335708bc-083d-4ce1-9158-cc61dcac93da-istiod-ca-cert\") pod \"e2e-worker-w1-d516c4-openshift-default-6bd584745-bs4lr\" (UID: \"335708bc-083d-4ce1-9158-cc61dcac93da\") " pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-bs4lr"
Oct 30 21:53:50.925023 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:50.924987    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/335708bc-083d-4ce1-9158-cc61dcac93da-workload-certs\") pod \"e2e-worker-w1-d516c4-openshift-default-6bd584745-bs4lr\" (UID: \"335708bc-083d-4ce1-9158-cc61dcac93da\") " pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-bs4lr"
Oct 30 21:53:50.925023 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:50.925024    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n865n\" (UniqueName: \"kubernetes.io/projected/335708bc-083d-4ce1-9158-cc61dcac93da-kube-api-access-n865n\") pod \"e2e-worker-w1-d516c4-openshift-default-6bd584745-bs4lr\" (UID: \"335708bc-083d-4ce1-9158-cc61dcac93da\") " pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-bs4lr"
Oct 30 21:53:50.925271 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:50.925057    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/335708bc-083d-4ce1-9158-cc61dcac93da-istio-data\") pod \"e2e-worker-w1-d516c4-openshift-default-6bd584745-bs4lr\" (UID: \"335708bc-083d-4ce1-9158-cc61dcac93da\") " pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-bs4lr"
Oct 30 21:53:50.925271 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:50.925086    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/335708bc-083d-4ce1-9158-cc61dcac93da-istio-podinfo\") pod \"e2e-worker-w1-d516c4-openshift-default-6bd584745-bs4lr\" (UID: \"335708bc-083d-4ce1-9158-cc61dcac93da\") " pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-bs4lr"
Oct 30 21:53:50.925271 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:50.925116    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/335708bc-083d-4ce1-9158-cc61dcac93da-credential-socket\") pod \"e2e-worker-w1-d516c4-openshift-default-6bd584745-bs4lr\" (UID: \"335708bc-083d-4ce1-9158-cc61dcac93da\") " pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-bs4lr"
Oct 30 21:53:50.925271 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:50.925177    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/335708bc-083d-4ce1-9158-cc61dcac93da-istio-token\") pod \"e2e-worker-w1-d516c4-openshift-default-6bd584745-bs4lr\" (UID: \"335708bc-083d-4ce1-9158-cc61dcac93da\") " pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-bs4lr"
Oct 30 21:53:50.925831 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:50.925805    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/335708bc-083d-4ce1-9158-cc61dcac93da-istio-envoy\") pod \"e2e-worker-w1-d516c4-openshift-default-6bd584745-bs4lr\" (UID: \"335708bc-083d-4ce1-9158-cc61dcac93da\") " pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-bs4lr"
Oct 30 21:53:50.925831 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:50.925493    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/335708bc-083d-4ce1-9158-cc61dcac93da-workload-certs\") pod \"e2e-worker-w1-d516c4-openshift-default-6bd584745-bs4lr\" (UID: \"335708bc-083d-4ce1-9158-cc61dcac93da\") " pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-bs4lr"
Oct 30 21:53:50.926056 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:50.925602    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/335708bc-083d-4ce1-9158-cc61dcac93da-credential-socket\") pod \"e2e-worker-w1-d516c4-openshift-default-6bd584745-bs4lr\" (UID: \"335708bc-083d-4ce1-9158-cc61dcac93da\") " pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-bs4lr"
Oct 30 21:53:50.926056 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:50.925895    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/335708bc-083d-4ce1-9158-cc61dcac93da-workload-socket\") pod \"e2e-worker-w1-d516c4-openshift-default-6bd584745-bs4lr\" (UID: \"335708bc-083d-4ce1-9158-cc61dcac93da\") " pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-bs4lr"
Oct 30 21:53:50.926056 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:50.925507    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/335708bc-083d-4ce1-9158-cc61dcac93da-istio-data\") pod \"e2e-worker-w1-d516c4-openshift-default-6bd584745-bs4lr\" (UID: \"335708bc-083d-4ce1-9158-cc61dcac93da\") " pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-bs4lr"
Oct 30 21:53:50.926056 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:50.925968    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/335708bc-083d-4ce1-9158-cc61dcac93da-istiod-ca-cert\") pod \"e2e-worker-w1-d516c4-openshift-default-6bd584745-bs4lr\" (UID: \"335708bc-083d-4ce1-9158-cc61dcac93da\") " pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-bs4lr"
Oct 30 21:53:50.926297 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:50.926271    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/335708bc-083d-4ce1-9158-cc61dcac93da-workload-socket\") pod \"e2e-worker-w1-d516c4-openshift-default-6bd584745-bs4lr\" (UID: \"335708bc-083d-4ce1-9158-cc61dcac93da\") " pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-bs4lr"
Oct 30 21:53:50.926583 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:50.926562    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/335708bc-083d-4ce1-9158-cc61dcac93da-istiod-ca-cert\") pod \"e2e-worker-w1-d516c4-openshift-default-6bd584745-bs4lr\" (UID: \"335708bc-083d-4ce1-9158-cc61dcac93da\") " pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-bs4lr"
Oct 30 21:53:50.928442 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:50.928424    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/335708bc-083d-4ce1-9158-cc61dcac93da-istio-podinfo\") pod \"e2e-worker-w1-d516c4-openshift-default-6bd584745-bs4lr\" (UID: \"335708bc-083d-4ce1-9158-cc61dcac93da\") " pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-bs4lr"
Oct 30 21:53:50.928664 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:50.928647    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/335708bc-083d-4ce1-9158-cc61dcac93da-istio-envoy\") pod \"e2e-worker-w1-d516c4-openshift-default-6bd584745-bs4lr\" (UID: \"335708bc-083d-4ce1-9158-cc61dcac93da\") " pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-bs4lr"
Oct 30 21:53:50.987868 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:50.987791    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/335708bc-083d-4ce1-9158-cc61dcac93da-istio-token\") pod \"e2e-worker-w1-d516c4-openshift-default-6bd584745-bs4lr\" (UID: \"335708bc-083d-4ce1-9158-cc61dcac93da\") " pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-bs4lr"
Oct 30 21:53:50.988421 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:50.988400    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n865n\" (UniqueName: \"kubernetes.io/projected/335708bc-083d-4ce1-9158-cc61dcac93da-kube-api-access-n865n\") pod \"e2e-worker-w1-d516c4-openshift-default-6bd584745-bs4lr\" (UID: \"335708bc-083d-4ce1-9158-cc61dcac93da\") " pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-bs4lr"
Oct 30 21:53:51.057840 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:51.057449    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-bs4lr"
Oct 30 21:53:51.100375 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:51.100342    4473 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-j5tqr"]
Oct 30 21:53:51.105816 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:51.105009    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-j5tqr"
Oct 30 21:53:51.128668 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:51.128633    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0f0fbc43-d98b-4b7c-872b-7978f76650ef-istio-envoy\") pod \"e2e-worker-w3-0f42cb-openshift-default-584f4c4865-j5tqr\" (UID: \"0f0fbc43-d98b-4b7c-872b-7978f76650ef\") " pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-j5tqr"
Oct 30 21:53:51.128823 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:51.128702    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0f0fbc43-d98b-4b7c-872b-7978f76650ef-istio-data\") pod \"e2e-worker-w3-0f42cb-openshift-default-584f4c4865-j5tqr\" (UID: \"0f0fbc43-d98b-4b7c-872b-7978f76650ef\") " pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-j5tqr"
Oct 30 21:53:51.128823 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:51.128764    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0f0fbc43-d98b-4b7c-872b-7978f76650ef-credential-socket\") pod \"e2e-worker-w3-0f42cb-openshift-default-584f4c4865-j5tqr\" (UID: \"0f0fbc43-d98b-4b7c-872b-7978f76650ef\") " pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-j5tqr"
Oct 30 21:53:51.128823 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:51.128791    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0f0fbc43-d98b-4b7c-872b-7978f76650ef-workload-certs\") pod \"e2e-worker-w3-0f42cb-openshift-default-584f4c4865-j5tqr\" (UID: \"0f0fbc43-d98b-4b7c-872b-7978f76650ef\") " pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-j5tqr"
Oct 30 21:53:51.128823 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:51.128818    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0f0fbc43-d98b-4b7c-872b-7978f76650ef-istio-token\") pod \"e2e-worker-w3-0f42cb-openshift-default-584f4c4865-j5tqr\" (UID: \"0f0fbc43-d98b-4b7c-872b-7978f76650ef\") " pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-j5tqr"
Oct 30 21:53:51.129066 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:51.128847    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0f0fbc43-d98b-4b7c-872b-7978f76650ef-istiod-ca-cert\") pod \"e2e-worker-w3-0f42cb-openshift-default-584f4c4865-j5tqr\" (UID: \"0f0fbc43-d98b-4b7c-872b-7978f76650ef\") " pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-j5tqr"
Oct 30 21:53:51.129066 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:51.128887    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0f0fbc43-d98b-4b7c-872b-7978f76650ef-istio-podinfo\") pod \"e2e-worker-w3-0f42cb-openshift-default-584f4c4865-j5tqr\" (UID: \"0f0fbc43-d98b-4b7c-872b-7978f76650ef\") " pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-j5tqr"
Oct 30 21:53:51.129066 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:51.129025    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0f0fbc43-d98b-4b7c-872b-7978f76650ef-workload-socket\") pod \"e2e-worker-w3-0f42cb-openshift-default-584f4c4865-j5tqr\" (UID: \"0f0fbc43-d98b-4b7c-872b-7978f76650ef\") " pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-j5tqr"
Oct 30 21:53:51.129208 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:51.129081    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bd2bj\" (UniqueName: \"kubernetes.io/projected/0f0fbc43-d98b-4b7c-872b-7978f76650ef-kube-api-access-bd2bj\") pod \"e2e-worker-w3-0f42cb-openshift-default-584f4c4865-j5tqr\" (UID: \"0f0fbc43-d98b-4b7c-872b-7978f76650ef\") " pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-j5tqr"
Oct 30 21:53:51.159204 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:51.159145    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w3-0f42cb-openshift-default-dockercfg-7vjjp\""
Oct 30 21:53:51.173818 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:51.173381    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-j5tqr"]
Oct 30 21:53:51.230473 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:51.230435    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bd2bj\" (UniqueName: \"kubernetes.io/projected/0f0fbc43-d98b-4b7c-872b-7978f76650ef-kube-api-access-bd2bj\") pod \"e2e-worker-w3-0f42cb-openshift-default-584f4c4865-j5tqr\" (UID: \"0f0fbc43-d98b-4b7c-872b-7978f76650ef\") " pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-j5tqr"
Oct 30 21:53:51.230628 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:51.230498    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0f0fbc43-d98b-4b7c-872b-7978f76650ef-istio-envoy\") pod \"e2e-worker-w3-0f42cb-openshift-default-584f4c4865-j5tqr\" (UID: \"0f0fbc43-d98b-4b7c-872b-7978f76650ef\") " pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-j5tqr"
Oct 30 21:53:51.230628 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:51.230536    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0f0fbc43-d98b-4b7c-872b-7978f76650ef-istio-data\") pod \"e2e-worker-w3-0f42cb-openshift-default-584f4c4865-j5tqr\" (UID: \"0f0fbc43-d98b-4b7c-872b-7978f76650ef\") " pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-j5tqr"
Oct 30 21:53:51.230628 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:51.230579    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0f0fbc43-d98b-4b7c-872b-7978f76650ef-credential-socket\") pod \"e2e-worker-w3-0f42cb-openshift-default-584f4c4865-j5tqr\" (UID: \"0f0fbc43-d98b-4b7c-872b-7978f76650ef\") " pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-j5tqr"
Oct 30 21:53:51.230628 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:51.230599    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0f0fbc43-d98b-4b7c-872b-7978f76650ef-workload-certs\") pod \"e2e-worker-w3-0f42cb-openshift-default-584f4c4865-j5tqr\" (UID: \"0f0fbc43-d98b-4b7c-872b-7978f76650ef\") " pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-j5tqr"
Oct 30 21:53:51.230628 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:51.230626    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0f0fbc43-d98b-4b7c-872b-7978f76650ef-istio-token\") pod \"e2e-worker-w3-0f42cb-openshift-default-584f4c4865-j5tqr\" (UID: \"0f0fbc43-d98b-4b7c-872b-7978f76650ef\") " pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-j5tqr"
Oct 30 21:53:51.230875 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:51.230650    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0f0fbc43-d98b-4b7c-872b-7978f76650ef-istiod-ca-cert\") pod \"e2e-worker-w3-0f42cb-openshift-default-584f4c4865-j5tqr\" (UID: \"0f0fbc43-d98b-4b7c-872b-7978f76650ef\") " pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-j5tqr"
Oct 30 21:53:51.230875 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:51.230681    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0f0fbc43-d98b-4b7c-872b-7978f76650ef-istio-podinfo\") pod \"e2e-worker-w3-0f42cb-openshift-default-584f4c4865-j5tqr\" (UID: \"0f0fbc43-d98b-4b7c-872b-7978f76650ef\") " pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-j5tqr"
Oct 30 21:53:51.230875 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:51.230719    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0f0fbc43-d98b-4b7c-872b-7978f76650ef-workload-socket\") pod \"e2e-worker-w3-0f42cb-openshift-default-584f4c4865-j5tqr\" (UID: \"0f0fbc43-d98b-4b7c-872b-7978f76650ef\") " pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-j5tqr"
Oct 30 21:53:51.231074 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:51.231040    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0f0fbc43-d98b-4b7c-872b-7978f76650ef-workload-certs\") pod \"e2e-worker-w3-0f42cb-openshift-default-584f4c4865-j5tqr\" (UID: \"0f0fbc43-d98b-4b7c-872b-7978f76650ef\") " pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-j5tqr"
Oct 30 21:53:51.231138 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:51.231095    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0f0fbc43-d98b-4b7c-872b-7978f76650ef-credential-socket\") pod \"e2e-worker-w3-0f42cb-openshift-default-584f4c4865-j5tqr\" (UID: \"0f0fbc43-d98b-4b7c-872b-7978f76650ef\") " pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-j5tqr"
Oct 30 21:53:51.231237 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:51.231217    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0f0fbc43-d98b-4b7c-872b-7978f76650ef-istio-data\") pod \"e2e-worker-w3-0f42cb-openshift-default-584f4c4865-j5tqr\" (UID: \"0f0fbc43-d98b-4b7c-872b-7978f76650ef\") " pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-j5tqr"
Oct 30 21:53:51.231288 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:51.231249    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0f0fbc43-d98b-4b7c-872b-7978f76650ef-workload-socket\") pod \"e2e-worker-w3-0f42cb-openshift-default-584f4c4865-j5tqr\" (UID: \"0f0fbc43-d98b-4b7c-872b-7978f76650ef\") " pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-j5tqr"
Oct 30 21:53:51.231420 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:51.231398    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0f0fbc43-d98b-4b7c-872b-7978f76650ef-istiod-ca-cert\") pod \"e2e-worker-w3-0f42cb-openshift-default-584f4c4865-j5tqr\" (UID: \"0f0fbc43-d98b-4b7c-872b-7978f76650ef\") " pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-j5tqr"
Oct 30 21:53:51.233679 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:51.233638    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0f0fbc43-d98b-4b7c-872b-7978f76650ef-istio-envoy\") pod \"e2e-worker-w3-0f42cb-openshift-default-584f4c4865-j5tqr\" (UID: \"0f0fbc43-d98b-4b7c-872b-7978f76650ef\") " pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-j5tqr"
Oct 30 21:53:51.233960 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:51.233909    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0f0fbc43-d98b-4b7c-872b-7978f76650ef-istio-podinfo\") pod \"e2e-worker-w3-0f42cb-openshift-default-584f4c4865-j5tqr\" (UID: \"0f0fbc43-d98b-4b7c-872b-7978f76650ef\") " pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-j5tqr"
Oct 30 21:53:51.289781 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:51.289697    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0f0fbc43-d98b-4b7c-872b-7978f76650ef-istio-token\") pod \"e2e-worker-w3-0f42cb-openshift-default-584f4c4865-j5tqr\" (UID: \"0f0fbc43-d98b-4b7c-872b-7978f76650ef\") " pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-j5tqr"
Oct 30 21:53:51.289912 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:51.289820    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bd2bj\" (UniqueName: \"kubernetes.io/projected/0f0fbc43-d98b-4b7c-872b-7978f76650ef-kube-api-access-bd2bj\") pod \"e2e-worker-w3-0f42cb-openshift-default-584f4c4865-j5tqr\" (UID: \"0f0fbc43-d98b-4b7c-872b-7978f76650ef\") " pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-j5tqr"
Oct 30 21:53:51.386416 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:51.386380    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-xct57" event={"ID":"893a5438-3b54-4d77-8675-00260980c0f6","Type":"ContainerStarted","Data":"21ac28e226e4fd387318bd11bb97c8354829aaf45ba244118fbd6ac71dfb1b36"}
Oct 30 21:53:51.386599 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:51.386442    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-xct57" event={"ID":"893a5438-3b54-4d77-8675-00260980c0f6","Type":"ContainerStarted","Data":"f9e49bacba3d0a169bae3b97fd3bdafefb603f38a3fa86113a62ef474dadf00b"}
Oct 30 21:53:51.430914 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:51.430876    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-j5tqr"
Oct 30 21:53:51.492755 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:51.488561    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-bs4lr"]
Oct 30 21:53:51.493530 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:51.493493    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:53:51.493612 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:51.493581    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:53:51.493674 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:51.493627    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:53:51.757121 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:51.756827    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-qng2w"
Oct 30 21:53:51.822959 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:51.822885    4473 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-xct57" podStartSLOduration=2.822868409 podStartE2EDuration="2.822868409s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:53:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:53:51.51447595 +0000 UTC m=+677.287844438" watchObservedRunningTime="2026-10-30 21:53:51.822868409 +0000 UTC m=+677.596236896"
Oct 30 21:53:51.823766 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:51.823746    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-j5tqr"]
Oct 30 21:53:51.825732 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:53:51.825692    4473 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f0fbc43_d98b_4b7c_872b_7978f76650ef.slice/crio-7bf5c3ee7ea91f054f19d55b7bf77d8b53de7326111d8186516c06a50f58be26 WatchSource:0}: Error finding container 7bf5c3ee7ea91f054f19d55b7bf77d8b53de7326111d8186516c06a50f58be26: Status 404 returned error can't find the container with id 7bf5c3ee7ea91f054f19d55b7bf77d8b53de7326111d8186516c06a50f58be26
Oct 30 21:53:51.828947 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:51.828888    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:53:51.829042 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:51.829018    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:53:51.829101 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:51.829072    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:53:52.371415 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:52.371379    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-xct57"
Oct 30 21:53:52.377302 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:52.377274    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-xct57"
Oct 30 21:53:52.393983 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:52.393941    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-j5tqr" event={"ID":"0f0fbc43-d98b-4b7c-872b-7978f76650ef","Type":"ContainerStarted","Data":"3b0fe4884f0af7c72480e3c91905a0dd01fe4f95e9c9ba8c7665ad3c1f926e4c"}
Oct 30 21:53:52.393983 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:52.394002    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-j5tqr" event={"ID":"0f0fbc43-d98b-4b7c-872b-7978f76650ef","Type":"ContainerStarted","Data":"7bf5c3ee7ea91f054f19d55b7bf77d8b53de7326111d8186516c06a50f58be26"}
Oct 30 21:53:52.395638 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:52.395611    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-bs4lr" event={"ID":"335708bc-083d-4ce1-9158-cc61dcac93da","Type":"ContainerStarted","Data":"f8e22d2edf99ab821c2b98a87d18effbb6bb9b17a0e4ac6ec2693fd3826b85dd"}
Oct 30 21:53:52.395773 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:52.395655    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-bs4lr" event={"ID":"335708bc-083d-4ce1-9158-cc61dcac93da","Type":"ContainerStarted","Data":"b905be3188056fdd6bccfff53c5254be59a0cd70fa04a8df940ea3d5adc15e07"}
Oct 30 21:53:52.395840 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:52.395800    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-qng2w"
Oct 30 21:53:52.395888 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:52.395847    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-xct57"
Oct 30 21:53:52.396704 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:52.396685    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-qng2w"
Oct 30 21:53:52.396849 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:52.396836    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-xct57"
Oct 30 21:53:52.431178 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:52.431148    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-j5tqr"
Oct 30 21:53:52.436457 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:52.436430    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-j5tqr"
Oct 30 21:53:52.743393 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:52.743290    4473 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-bs4lr" podStartSLOduration=2.743277003 podStartE2EDuration="2.743277003s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:53:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:53:52.742303709 +0000 UTC m=+678.515672197" watchObservedRunningTime="2026-10-30 21:53:52.743277003 +0000 UTC m=+678.516645490"
Oct 30 21:53:52.870953 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:52.870869    4473 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-j5tqr" podStartSLOduration=1.8708487790000001 podStartE2EDuration="1.870848779s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:53:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:53:52.861767186 +0000 UTC m=+678.635135690" watchObservedRunningTime="2026-10-30 21:53:52.870848779 +0000 UTC m=+678.644217266"
Oct 30 21:53:53.058451 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:53.058368    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-bs4lr"
Oct 30 21:53:53.065156 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:53.065130    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-bs4lr"
Oct 30 21:53:53.401550 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:53.401513    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-j5tqr"
Oct 30 21:53:53.401757 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:53.401564    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-bs4lr"
Oct 30 21:53:53.402205 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:53.402182    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-j5tqr"
Oct 30 21:53:53.402479 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:53.402467    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-bs4lr"
Oct 30 21:53:58.905205 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:58.905167    4473 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w2-0c99b1-775568968c-tltkr"]
Oct 30 21:53:58.909209 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:58.909190    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-0c99b1-775568968c-tltkr"
Oct 30 21:53:58.963978 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:58.963944    4473 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w2-0c99b1\""
Oct 30 21:53:59.018340 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:59.014812    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w2-0c99b1-dockercfg-xrxqn\""
Oct 30 21:53:59.024979 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:59.019333    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w2-0c99b1-775568968c-tltkr"]
Oct 30 21:53:59.052035 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:59.051770    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lwnbv\" (UniqueName: \"kubernetes.io/projected/f260821f-0996-4717-9f4f-c30f41c238cb-kube-api-access-lwnbv\") pod \"payload-pre-processing-e2e-worker-w2-0c99b1-775568968c-tltkr\" (UID: \"f260821f-0996-4717-9f4f-c30f41c238cb\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-0c99b1-775568968c-tltkr"
Oct 30 21:53:59.052035 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:59.051866    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f260821f-0996-4717-9f4f-c30f41c238cb-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w2-0c99b1-775568968c-tltkr\" (UID: \"f260821f-0996-4717-9f4f-c30f41c238cb\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-0c99b1-775568968c-tltkr"
Oct 30 21:53:59.067309 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:59.067268    4473 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w2-0c99b1-64b58dc987-5h7bc"]
Oct 30 21:53:59.071013 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:59.070988    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w2-0c99b1-64b58dc987-5h7bc"
Oct 30 21:53:59.095423 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:59.095388    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w2-0c99b1-64b58dc987-5h7bc"]
Oct 30 21:53:59.152688 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:59.152655    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nxvzp\" (UniqueName: \"kubernetes.io/projected/405c4ba4-c940-45e9-bcb9-484428bf5c5b-kube-api-access-nxvzp\") pod \"payload-processing-e2e-worker-w2-0c99b1-64b58dc987-5h7bc\" (UID: \"405c4ba4-c940-45e9-bcb9-484428bf5c5b\") " pod="openshift-ingress/payload-processing-e2e-worker-w2-0c99b1-64b58dc987-5h7bc"
Oct 30 21:53:59.152880 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:59.152701    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/405c4ba4-c940-45e9-bcb9-484428bf5c5b-plugins-config-volume\") pod \"payload-processing-e2e-worker-w2-0c99b1-64b58dc987-5h7bc\" (UID: \"405c4ba4-c940-45e9-bcb9-484428bf5c5b\") " pod="openshift-ingress/payload-processing-e2e-worker-w2-0c99b1-64b58dc987-5h7bc"
Oct 30 21:53:59.152880 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:59.152788    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lwnbv\" (UniqueName: \"kubernetes.io/projected/f260821f-0996-4717-9f4f-c30f41c238cb-kube-api-access-lwnbv\") pod \"payload-pre-processing-e2e-worker-w2-0c99b1-775568968c-tltkr\" (UID: \"f260821f-0996-4717-9f4f-c30f41c238cb\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-0c99b1-775568968c-tltkr"
Oct 30 21:53:59.152880 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:59.152839    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f260821f-0996-4717-9f4f-c30f41c238cb-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w2-0c99b1-775568968c-tltkr\" (UID: \"f260821f-0996-4717-9f4f-c30f41c238cb\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-0c99b1-775568968c-tltkr"
Oct 30 21:53:59.153564 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:59.153545    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f260821f-0996-4717-9f4f-c30f41c238cb-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w2-0c99b1-775568968c-tltkr\" (UID: \"f260821f-0996-4717-9f4f-c30f41c238cb\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-0c99b1-775568968c-tltkr"
Oct 30 21:53:59.210981 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:59.210898    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lwnbv\" (UniqueName: \"kubernetes.io/projected/f260821f-0996-4717-9f4f-c30f41c238cb-kube-api-access-lwnbv\") pod \"payload-pre-processing-e2e-worker-w2-0c99b1-775568968c-tltkr\" (UID: \"f260821f-0996-4717-9f4f-c30f41c238cb\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-0c99b1-775568968c-tltkr"
Oct 30 21:53:59.222644 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:59.222616    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-0c99b1-775568968c-tltkr"
Oct 30 21:53:59.253866 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:59.253831    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nxvzp\" (UniqueName: \"kubernetes.io/projected/405c4ba4-c940-45e9-bcb9-484428bf5c5b-kube-api-access-nxvzp\") pod \"payload-processing-e2e-worker-w2-0c99b1-64b58dc987-5h7bc\" (UID: \"405c4ba4-c940-45e9-bcb9-484428bf5c5b\") " pod="openshift-ingress/payload-processing-e2e-worker-w2-0c99b1-64b58dc987-5h7bc"
Oct 30 21:53:59.254053 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:59.253884    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/405c4ba4-c940-45e9-bcb9-484428bf5c5b-plugins-config-volume\") pod \"payload-processing-e2e-worker-w2-0c99b1-64b58dc987-5h7bc\" (UID: \"405c4ba4-c940-45e9-bcb9-484428bf5c5b\") " pod="openshift-ingress/payload-processing-e2e-worker-w2-0c99b1-64b58dc987-5h7bc"
Oct 30 21:53:59.254626 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:59.254604    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/405c4ba4-c940-45e9-bcb9-484428bf5c5b-plugins-config-volume\") pod \"payload-processing-e2e-worker-w2-0c99b1-64b58dc987-5h7bc\" (UID: \"405c4ba4-c940-45e9-bcb9-484428bf5c5b\") " pod="openshift-ingress/payload-processing-e2e-worker-w2-0c99b1-64b58dc987-5h7bc"
Oct 30 21:53:59.312221 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:59.312188    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nxvzp\" (UniqueName: \"kubernetes.io/projected/405c4ba4-c940-45e9-bcb9-484428bf5c5b-kube-api-access-nxvzp\") pod \"payload-processing-e2e-worker-w2-0c99b1-64b58dc987-5h7bc\" (UID: \"405c4ba4-c940-45e9-bcb9-484428bf5c5b\") " pod="openshift-ingress/payload-processing-e2e-worker-w2-0c99b1-64b58dc987-5h7bc"
Oct 30 21:53:59.384353 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:59.384308    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w2-0c99b1-64b58dc987-5h7bc"
Oct 30 21:53:59.602668 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:59.602636    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w2-0c99b1-775568968c-tltkr"]
Oct 30 21:53:59.604411 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:53:59.604382    4473 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf260821f_0996_4717_9f4f_c30f41c238cb.slice/crio-c7535313f67eb48cf6677a4703d9e97c47975f86618b7a8440eaf3383f7dd2fc WatchSource:0}: Error finding container c7535313f67eb48cf6677a4703d9e97c47975f86618b7a8440eaf3383f7dd2fc: Status 404 returned error can't find the container with id c7535313f67eb48cf6677a4703d9e97c47975f86618b7a8440eaf3383f7dd2fc
Oct 30 21:53:59.760807 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:53:59.760779    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w2-0c99b1-64b58dc987-5h7bc"]
Oct 30 21:53:59.762566 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:53:59.762541    4473 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod405c4ba4_c940_45e9_bcb9_484428bf5c5b.slice/crio-28fcbe78525a9b81063cec44ffa7d24b25e38ea09594d2bbd07976374d0b1c10 WatchSource:0}: Error finding container 28fcbe78525a9b81063cec44ffa7d24b25e38ea09594d2bbd07976374d0b1c10: Status 404 returned error can't find the container with id 28fcbe78525a9b81063cec44ffa7d24b25e38ea09594d2bbd07976374d0b1c10
Oct 30 21:54:00.156270 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:00.156223    4473 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-b0c55f-8457bc5476-5gwgt"]
Oct 30 21:54:00.160071 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:00.160047    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-b0c55f-8457bc5476-5gwgt"
Oct 30 21:54:00.213459 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:00.213425    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w4-b0c55f-serving-cert\""
Oct 30 21:54:00.222643 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:00.222615    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-b0c55f-8457bc5476-5gwgt"]
Oct 30 21:54:00.266487 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:00.266451    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/895a52c4-b3cd-4cb0-8e80-46f08414b29f-metrics-certs\") pod \"maas-api-e2e-worker-w4-b0c55f-8457bc5476-5gwgt\" (UID: \"895a52c4-b3cd-4cb0-8e80-46f08414b29f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-b0c55f-8457bc5476-5gwgt"
Oct 30 21:54:00.266675 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:00.266512    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/895a52c4-b3cd-4cb0-8e80-46f08414b29f-maas-api-tls\") pod \"maas-api-e2e-worker-w4-b0c55f-8457bc5476-5gwgt\" (UID: \"895a52c4-b3cd-4cb0-8e80-46f08414b29f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-b0c55f-8457bc5476-5gwgt"
Oct 30 21:54:00.266753 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:00.266683    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/895a52c4-b3cd-4cb0-8e80-46f08414b29f-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-b0c55f-8457bc5476-5gwgt\" (UID: \"895a52c4-b3cd-4cb0-8e80-46f08414b29f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-b0c55f-8457bc5476-5gwgt"
Oct 30 21:54:00.266830 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:00.266811    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5jb6b\" (UniqueName: \"kubernetes.io/projected/895a52c4-b3cd-4cb0-8e80-46f08414b29f-kube-api-access-5jb6b\") pod \"maas-api-e2e-worker-w4-b0c55f-8457bc5476-5gwgt\" (UID: \"895a52c4-b3cd-4cb0-8e80-46f08414b29f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-b0c55f-8457bc5476-5gwgt"
Oct 30 21:54:00.367882 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:00.367847    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5jb6b\" (UniqueName: \"kubernetes.io/projected/895a52c4-b3cd-4cb0-8e80-46f08414b29f-kube-api-access-5jb6b\") pod \"maas-api-e2e-worker-w4-b0c55f-8457bc5476-5gwgt\" (UID: \"895a52c4-b3cd-4cb0-8e80-46f08414b29f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-b0c55f-8457bc5476-5gwgt"
Oct 30 21:54:00.368100 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:00.367962    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/895a52c4-b3cd-4cb0-8e80-46f08414b29f-metrics-certs\") pod \"maas-api-e2e-worker-w4-b0c55f-8457bc5476-5gwgt\" (UID: \"895a52c4-b3cd-4cb0-8e80-46f08414b29f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-b0c55f-8457bc5476-5gwgt"
Oct 30 21:54:00.368100 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:00.368002    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/895a52c4-b3cd-4cb0-8e80-46f08414b29f-maas-api-tls\") pod \"maas-api-e2e-worker-w4-b0c55f-8457bc5476-5gwgt\" (UID: \"895a52c4-b3cd-4cb0-8e80-46f08414b29f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-b0c55f-8457bc5476-5gwgt"
Oct 30 21:54:00.368100 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:00.368026    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/895a52c4-b3cd-4cb0-8e80-46f08414b29f-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-b0c55f-8457bc5476-5gwgt\" (UID: \"895a52c4-b3cd-4cb0-8e80-46f08414b29f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-b0c55f-8457bc5476-5gwgt"
Oct 30 21:54:00.368235 ip-10-0-140-18 kubenswrapper[4473]: E1030 21:54:00.368142    4473 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w4-b0c55f-serving-cert: secret "maas-api-e2e-worker-w4-b0c55f-serving-cert" not found
Oct 30 21:54:00.368235 ip-10-0-140-18 kubenswrapper[4473]: E1030 21:54:00.368214    4473 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/895a52c4-b3cd-4cb0-8e80-46f08414b29f-maas-api-tls podName:895a52c4-b3cd-4cb0-8e80-46f08414b29f nodeName:}" failed. No retries permitted until 2026-10-30 21:54:00.868197503 +0000 UTC m=+686.641565967 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/895a52c4-b3cd-4cb0-8e80-46f08414b29f-maas-api-tls") pod "maas-api-e2e-worker-w4-b0c55f-8457bc5476-5gwgt" (UID: "895a52c4-b3cd-4cb0-8e80-46f08414b29f") : secret "maas-api-e2e-worker-w4-b0c55f-serving-cert" not found
Oct 30 21:54:00.368681 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:00.368665    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/895a52c4-b3cd-4cb0-8e80-46f08414b29f-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-b0c55f-8457bc5476-5gwgt\" (UID: \"895a52c4-b3cd-4cb0-8e80-46f08414b29f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-b0c55f-8457bc5476-5gwgt"
Oct 30 21:54:00.370658 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:00.370632    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/895a52c4-b3cd-4cb0-8e80-46f08414b29f-metrics-certs\") pod \"maas-api-e2e-worker-w4-b0c55f-8457bc5476-5gwgt\" (UID: \"895a52c4-b3cd-4cb0-8e80-46f08414b29f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-b0c55f-8457bc5476-5gwgt"
Oct 30 21:54:00.440230 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:00.440130    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-0c99b1-775568968c-tltkr" event={"ID":"f260821f-0996-4717-9f4f-c30f41c238cb","Type":"ContainerStarted","Data":"f98299dcc16d9ec252eb58e0ef0b602f9e042b5109b121256b81450a8f29302a"}
Oct 30 21:54:00.440230 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:00.440192    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-0c99b1-775568968c-tltkr" event={"ID":"f260821f-0996-4717-9f4f-c30f41c238cb","Type":"ContainerStarted","Data":"c7535313f67eb48cf6677a4703d9e97c47975f86618b7a8440eaf3383f7dd2fc"}
Oct 30 21:54:00.441041 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:00.440962    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-0c99b1-775568968c-tltkr"
Oct 30 21:54:00.442658 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:00.442631    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w2-0c99b1-64b58dc987-5h7bc" event={"ID":"405c4ba4-c940-45e9-bcb9-484428bf5c5b","Type":"ContainerStarted","Data":"01b67c114f1f80a3d77401e5a41c2dbea7e7cc1e1f2059a69a7093920ba28150"}
Oct 30 21:54:00.442785 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:00.442685    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w2-0c99b1-64b58dc987-5h7bc" event={"ID":"405c4ba4-c940-45e9-bcb9-484428bf5c5b","Type":"ContainerStarted","Data":"28fcbe78525a9b81063cec44ffa7d24b25e38ea09594d2bbd07976374d0b1c10"}
Oct 30 21:54:00.442785 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:00.442722    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w2-0c99b1-64b58dc987-5h7bc"
Oct 30 21:54:00.452494 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:00.452463    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5jb6b\" (UniqueName: \"kubernetes.io/projected/895a52c4-b3cd-4cb0-8e80-46f08414b29f-kube-api-access-5jb6b\") pod \"maas-api-e2e-worker-w4-b0c55f-8457bc5476-5gwgt\" (UID: \"895a52c4-b3cd-4cb0-8e80-46f08414b29f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-b0c55f-8457bc5476-5gwgt"
Oct 30 21:54:00.604258 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:00.604207    4473 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-0c99b1-775568968c-tltkr" podStartSLOduration=2.604191494 podStartE2EDuration="2.604191494s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:53:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:54:00.602901481 +0000 UTC m=+686.376269968" watchObservedRunningTime="2026-10-30 21:54:00.604191494 +0000 UTC m=+686.377559980"
Oct 30 21:54:00.764230 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:00.764082    4473 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w2-0c99b1-64b58dc987-5h7bc" podStartSLOduration=2.76406031 podStartE2EDuration="2.76406031s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:53:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:54:00.760381472 +0000 UTC m=+686.533749971" watchObservedRunningTime="2026-10-30 21:54:00.76406031 +0000 UTC m=+686.537428798"
Oct 30 21:54:00.874161 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:00.874121    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/895a52c4-b3cd-4cb0-8e80-46f08414b29f-maas-api-tls\") pod \"maas-api-e2e-worker-w4-b0c55f-8457bc5476-5gwgt\" (UID: \"895a52c4-b3cd-4cb0-8e80-46f08414b29f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-b0c55f-8457bc5476-5gwgt"
Oct 30 21:54:00.876789 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:00.876760    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/895a52c4-b3cd-4cb0-8e80-46f08414b29f-maas-api-tls\") pod \"maas-api-e2e-worker-w4-b0c55f-8457bc5476-5gwgt\" (UID: \"895a52c4-b3cd-4cb0-8e80-46f08414b29f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-b0c55f-8457bc5476-5gwgt"
Oct 30 21:54:01.079737 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:01.079646    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-b0c55f-8457bc5476-5gwgt"
Oct 30 21:54:01.462280 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:01.462253    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-b0c55f-8457bc5476-5gwgt"]
Oct 30 21:54:01.463855 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:54:01.463824    4473 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod895a52c4_b3cd_4cb0_8e80_46f08414b29f.slice/crio-5b426bfddcf39a292c9c2a1a2f0c3a5c54fba721718b74925316eaa3e3f01474 WatchSource:0}: Error finding container 5b426bfddcf39a292c9c2a1a2f0c3a5c54fba721718b74925316eaa3e3f01474: Status 404 returned error can't find the container with id 5b426bfddcf39a292c9c2a1a2f0c3a5c54fba721718b74925316eaa3e3f01474
Oct 30 21:54:02.456030 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:02.455978    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-b0c55f-8457bc5476-5gwgt" event={"ID":"895a52c4-b3cd-4cb0-8e80-46f08414b29f","Type":"ContainerStarted","Data":"aba9e873750983f6175d101713712392662f1320b48b139c2394c717a9203b84"}
Oct 30 21:54:02.456030 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:02.456038    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-b0c55f-8457bc5476-5gwgt" event={"ID":"895a52c4-b3cd-4cb0-8e80-46f08414b29f","Type":"ContainerStarted","Data":"5b426bfddcf39a292c9c2a1a2f0c3a5c54fba721718b74925316eaa3e3f01474"}
Oct 30 21:54:02.456293 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:02.456227    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-b0c55f-8457bc5476-5gwgt"
Oct 30 21:54:02.581258 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:02.581200    4473 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-b0c55f-8457bc5476-5gwgt" podStartSLOduration=1.668241171 podStartE2EDuration="2.581182783s" totalImagesPullingTime="912.941612ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:54:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:54:02.579262223 +0000 UTC m=+688.352630710" watchObservedRunningTime="2026-10-30 21:54:02.581182783 +0000 UTC m=+688.354551270"
Oct 30 21:54:06.411100 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:06.411062    4473 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w5-9f8aea-df9ff57d7-k7mlk"]
Oct 30 21:54:06.415083 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:06.415063    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-9f8aea-df9ff57d7-k7mlk"
Oct 30 21:54:06.473308 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:06.473265    4473 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w5-9f8aea\""
Oct 30 21:54:06.523891 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:06.523853    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w5-9f8aea-dockercfg-qbpvx\""
Oct 30 21:54:06.526015 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:06.525991    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w5-9f8aea-df9ff57d7-k7mlk"]
Oct 30 21:54:06.533405 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:06.533357    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7665007f-2e07-47f7-ac42-82dfd53d85a1-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w5-9f8aea-df9ff57d7-k7mlk\" (UID: \"7665007f-2e07-47f7-ac42-82dfd53d85a1\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-9f8aea-df9ff57d7-k7mlk"
Oct 30 21:54:06.533693 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:06.533497    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d6bw8\" (UniqueName: \"kubernetes.io/projected/7665007f-2e07-47f7-ac42-82dfd53d85a1-kube-api-access-d6bw8\") pod \"payload-pre-processing-e2e-worker-w5-9f8aea-df9ff57d7-k7mlk\" (UID: \"7665007f-2e07-47f7-ac42-82dfd53d85a1\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-9f8aea-df9ff57d7-k7mlk"
Oct 30 21:54:06.634799 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:06.634763    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7665007f-2e07-47f7-ac42-82dfd53d85a1-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w5-9f8aea-df9ff57d7-k7mlk\" (UID: \"7665007f-2e07-47f7-ac42-82dfd53d85a1\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-9f8aea-df9ff57d7-k7mlk"
Oct 30 21:54:06.635046 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:06.634846    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d6bw8\" (UniqueName: \"kubernetes.io/projected/7665007f-2e07-47f7-ac42-82dfd53d85a1-kube-api-access-d6bw8\") pod \"payload-pre-processing-e2e-worker-w5-9f8aea-df9ff57d7-k7mlk\" (UID: \"7665007f-2e07-47f7-ac42-82dfd53d85a1\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-9f8aea-df9ff57d7-k7mlk"
Oct 30 21:54:06.635547 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:06.635524    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7665007f-2e07-47f7-ac42-82dfd53d85a1-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w5-9f8aea-df9ff57d7-k7mlk\" (UID: \"7665007f-2e07-47f7-ac42-82dfd53d85a1\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-9f8aea-df9ff57d7-k7mlk"
Oct 30 21:54:06.694652 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:06.694585    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d6bw8\" (UniqueName: \"kubernetes.io/projected/7665007f-2e07-47f7-ac42-82dfd53d85a1-kube-api-access-d6bw8\") pod \"payload-pre-processing-e2e-worker-w5-9f8aea-df9ff57d7-k7mlk\" (UID: \"7665007f-2e07-47f7-ac42-82dfd53d85a1\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-9f8aea-df9ff57d7-k7mlk"
Oct 30 21:54:06.729106 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:06.729067    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-9f8aea-df9ff57d7-k7mlk"
Oct 30 21:54:07.109113 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:07.109083    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w5-9f8aea-df9ff57d7-k7mlk"]
Oct 30 21:54:07.110970 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:54:07.110940    4473 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7665007f_2e07_47f7_ac42_82dfd53d85a1.slice/crio-c97dc21c154a651d5dee7eb506afd85abb479c66b8c64f45011dfee87975df2c WatchSource:0}: Error finding container c97dc21c154a651d5dee7eb506afd85abb479c66b8c64f45011dfee87975df2c: Status 404 returned error can't find the container with id c97dc21c154a651d5dee7eb506afd85abb479c66b8c64f45011dfee87975df2c
Oct 30 21:54:07.498068 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:07.498030    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-9f8aea-df9ff57d7-k7mlk" event={"ID":"7665007f-2e07-47f7-ac42-82dfd53d85a1","Type":"ContainerStarted","Data":"533a86cb750ea2e47f0f08fd67a09ed43ebe877cddb816f28ed25be603352c5c"}
Oct 30 21:54:07.498554 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:07.498085    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-9f8aea-df9ff57d7-k7mlk" event={"ID":"7665007f-2e07-47f7-ac42-82dfd53d85a1","Type":"ContainerStarted","Data":"c97dc21c154a651d5dee7eb506afd85abb479c66b8c64f45011dfee87975df2c"}
Oct 30 21:54:07.498554 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:07.498116    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-9f8aea-df9ff57d7-k7mlk"
Oct 30 21:54:07.616369 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:07.616315    4473 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-9f8aea-df9ff57d7-k7mlk" podStartSLOduration=1.6162997300000002 podStartE2EDuration="1.61629973s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:54:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:54:07.615192273 +0000 UTC m=+693.388560763" watchObservedRunningTime="2026-10-30 21:54:07.61629973 +0000 UTC m=+693.389668237"
Oct 30 21:54:08.466477 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:08.466451    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-b0c55f-8457bc5476-5gwgt"
Oct 30 21:54:10.998167 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:10.998129    4473 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-d516c4-687f696c99-xjfqk"]
Oct 30 21:54:11.002291 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:11.002269    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-d516c4-687f696c99-xjfqk"
Oct 30 21:54:11.055732 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:11.055701    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w1-d516c4-serving-cert\""
Oct 30 21:54:11.062935 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:11.062880    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-d516c4-687f696c99-xjfqk"]
Oct 30 21:54:11.088185 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:11.088126    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/164e37d6-3d6f-4210-974d-6e6016c97b40-openshift-service-ca\") pod \"maas-api-e2e-worker-w1-d516c4-687f696c99-xjfqk\" (UID: \"164e37d6-3d6f-4210-974d-6e6016c97b40\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-d516c4-687f696c99-xjfqk"
Oct 30 21:54:11.088185 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:11.088179    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/164e37d6-3d6f-4210-974d-6e6016c97b40-metrics-certs\") pod \"maas-api-e2e-worker-w1-d516c4-687f696c99-xjfqk\" (UID: \"164e37d6-3d6f-4210-974d-6e6016c97b40\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-d516c4-687f696c99-xjfqk"
Oct 30 21:54:11.088441 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:11.088319    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/164e37d6-3d6f-4210-974d-6e6016c97b40-maas-api-tls\") pod \"maas-api-e2e-worker-w1-d516c4-687f696c99-xjfqk\" (UID: \"164e37d6-3d6f-4210-974d-6e6016c97b40\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-d516c4-687f696c99-xjfqk"
Oct 30 21:54:11.088441 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:11.088361    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2l68v\" (UniqueName: \"kubernetes.io/projected/164e37d6-3d6f-4210-974d-6e6016c97b40-kube-api-access-2l68v\") pod \"maas-api-e2e-worker-w1-d516c4-687f696c99-xjfqk\" (UID: \"164e37d6-3d6f-4210-974d-6e6016c97b40\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-d516c4-687f696c99-xjfqk"
Oct 30 21:54:11.189474 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:11.189431    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/164e37d6-3d6f-4210-974d-6e6016c97b40-openshift-service-ca\") pod \"maas-api-e2e-worker-w1-d516c4-687f696c99-xjfqk\" (UID: \"164e37d6-3d6f-4210-974d-6e6016c97b40\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-d516c4-687f696c99-xjfqk"
Oct 30 21:54:11.189474 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:11.189470    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/164e37d6-3d6f-4210-974d-6e6016c97b40-metrics-certs\") pod \"maas-api-e2e-worker-w1-d516c4-687f696c99-xjfqk\" (UID: \"164e37d6-3d6f-4210-974d-6e6016c97b40\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-d516c4-687f696c99-xjfqk"
Oct 30 21:54:11.189746 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:11.189600    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/164e37d6-3d6f-4210-974d-6e6016c97b40-maas-api-tls\") pod \"maas-api-e2e-worker-w1-d516c4-687f696c99-xjfqk\" (UID: \"164e37d6-3d6f-4210-974d-6e6016c97b40\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-d516c4-687f696c99-xjfqk"
Oct 30 21:54:11.189746 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:11.189642    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2l68v\" (UniqueName: \"kubernetes.io/projected/164e37d6-3d6f-4210-974d-6e6016c97b40-kube-api-access-2l68v\") pod \"maas-api-e2e-worker-w1-d516c4-687f696c99-xjfqk\" (UID: \"164e37d6-3d6f-4210-974d-6e6016c97b40\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-d516c4-687f696c99-xjfqk"
Oct 30 21:54:11.189848 ip-10-0-140-18 kubenswrapper[4473]: E1030 21:54:11.189775    4473 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w1-d516c4-serving-cert: secret "maas-api-e2e-worker-w1-d516c4-serving-cert" not found
Oct 30 21:54:11.189899 ip-10-0-140-18 kubenswrapper[4473]: E1030 21:54:11.189854    4473 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/164e37d6-3d6f-4210-974d-6e6016c97b40-maas-api-tls podName:164e37d6-3d6f-4210-974d-6e6016c97b40 nodeName:}" failed. No retries permitted until 2026-10-30 21:54:11.689831114 +0000 UTC m=+697.463199584 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/164e37d6-3d6f-4210-974d-6e6016c97b40-maas-api-tls") pod "maas-api-e2e-worker-w1-d516c4-687f696c99-xjfqk" (UID: "164e37d6-3d6f-4210-974d-6e6016c97b40") : secret "maas-api-e2e-worker-w1-d516c4-serving-cert" not found
Oct 30 21:54:11.190296 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:11.190272    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/164e37d6-3d6f-4210-974d-6e6016c97b40-openshift-service-ca\") pod \"maas-api-e2e-worker-w1-d516c4-687f696c99-xjfqk\" (UID: \"164e37d6-3d6f-4210-974d-6e6016c97b40\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-d516c4-687f696c99-xjfqk"
Oct 30 21:54:11.192766 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:11.192742    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/164e37d6-3d6f-4210-974d-6e6016c97b40-metrics-certs\") pod \"maas-api-e2e-worker-w1-d516c4-687f696c99-xjfqk\" (UID: \"164e37d6-3d6f-4210-974d-6e6016c97b40\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-d516c4-687f696c99-xjfqk"
Oct 30 21:54:11.247863 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:11.247834    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2l68v\" (UniqueName: \"kubernetes.io/projected/164e37d6-3d6f-4210-974d-6e6016c97b40-kube-api-access-2l68v\") pod \"maas-api-e2e-worker-w1-d516c4-687f696c99-xjfqk\" (UID: \"164e37d6-3d6f-4210-974d-6e6016c97b40\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-d516c4-687f696c99-xjfqk"
Oct 30 21:54:11.452414 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:11.452385    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w2-0c99b1-64b58dc987-5h7bc"
Oct 30 21:54:11.696207 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:11.696142    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/164e37d6-3d6f-4210-974d-6e6016c97b40-maas-api-tls\") pod \"maas-api-e2e-worker-w1-d516c4-687f696c99-xjfqk\" (UID: \"164e37d6-3d6f-4210-974d-6e6016c97b40\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-d516c4-687f696c99-xjfqk"
Oct 30 21:54:11.698782 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:11.698762    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/164e37d6-3d6f-4210-974d-6e6016c97b40-maas-api-tls\") pod \"maas-api-e2e-worker-w1-d516c4-687f696c99-xjfqk\" (UID: \"164e37d6-3d6f-4210-974d-6e6016c97b40\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-d516c4-687f696c99-xjfqk"
Oct 30 21:54:11.915946 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:11.915873    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-d516c4-687f696c99-xjfqk"
Oct 30 21:54:12.342571 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:12.342542    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-d516c4-687f696c99-xjfqk"]
Oct 30 21:54:12.343971 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:54:12.343945    4473 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod164e37d6_3d6f_4210_974d_6e6016c97b40.slice/crio-24f2d99b98c5c29398324201a7e405e8b5aef47b022079208106b0a1df77f017 WatchSource:0}: Error finding container 24f2d99b98c5c29398324201a7e405e8b5aef47b022079208106b0a1df77f017: Status 404 returned error can't find the container with id 24f2d99b98c5c29398324201a7e405e8b5aef47b022079208106b0a1df77f017
Oct 30 21:54:12.459016 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:12.458987    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-0c99b1-775568968c-tltkr"
Oct 30 21:54:12.535510 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:12.535477    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-d516c4-687f696c99-xjfqk" event={"ID":"164e37d6-3d6f-4210-974d-6e6016c97b40","Type":"ContainerStarted","Data":"24f2d99b98c5c29398324201a7e405e8b5aef47b022079208106b0a1df77f017"}
Oct 30 21:54:13.545227 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:13.545131    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-d516c4-687f696c99-xjfqk" event={"ID":"164e37d6-3d6f-4210-974d-6e6016c97b40","Type":"ContainerStarted","Data":"161e4dd435e6d28824acd86fa28f5ae11dd6313527b49c13db89e9abc1fe1ad3"}
Oct 30 21:54:13.545227 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:13.545199    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-d516c4-687f696c99-xjfqk"
Oct 30 21:54:13.662227 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:13.662176    4473 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-d516c4-687f696c99-xjfqk" podStartSLOduration=2.792682773 podStartE2EDuration="3.662160421s" totalImagesPullingTime="869.477648ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:54:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:54:13.661647157 +0000 UTC m=+699.435015645" watchObservedRunningTime="2026-10-30 21:54:13.662160421 +0000 UTC m=+699.435528908"
Oct 30 21:54:16.473213 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:16.473177    4473 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-0f42cb-566b854747-xhdwq"]
Oct 30 21:54:16.477018 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:16.477001    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w3-0f42cb-566b854747-xhdwq"
Oct 30 21:54:16.540490 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:16.540457    4473 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w3-0f42cb\""
Oct 30 21:54:16.564726 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:16.564695    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vfqj4\" (UniqueName: \"kubernetes.io/projected/17ff1c66-c309-45da-9843-b37262d288d6-kube-api-access-vfqj4\") pod \"payload-processing-e2e-worker-w3-0f42cb-566b854747-xhdwq\" (UID: \"17ff1c66-c309-45da-9843-b37262d288d6\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-0f42cb-566b854747-xhdwq"
Oct 30 21:54:16.564895 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:16.564756    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/17ff1c66-c309-45da-9843-b37262d288d6-plugins-config-volume\") pod \"payload-processing-e2e-worker-w3-0f42cb-566b854747-xhdwq\" (UID: \"17ff1c66-c309-45da-9843-b37262d288d6\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-0f42cb-566b854747-xhdwq"
Oct 30 21:54:16.594639 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:16.594606    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-0f42cb-566b854747-xhdwq"]
Oct 30 21:54:16.599390 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:16.599368    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w3-0f42cb-dockercfg-xp276\""
Oct 30 21:54:16.665840 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:16.665807    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vfqj4\" (UniqueName: \"kubernetes.io/projected/17ff1c66-c309-45da-9843-b37262d288d6-kube-api-access-vfqj4\") pod \"payload-processing-e2e-worker-w3-0f42cb-566b854747-xhdwq\" (UID: \"17ff1c66-c309-45da-9843-b37262d288d6\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-0f42cb-566b854747-xhdwq"
Oct 30 21:54:16.666050 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:16.665861    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/17ff1c66-c309-45da-9843-b37262d288d6-plugins-config-volume\") pod \"payload-processing-e2e-worker-w3-0f42cb-566b854747-xhdwq\" (UID: \"17ff1c66-c309-45da-9843-b37262d288d6\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-0f42cb-566b854747-xhdwq"
Oct 30 21:54:16.666607 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:16.666588    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/17ff1c66-c309-45da-9843-b37262d288d6-plugins-config-volume\") pod \"payload-processing-e2e-worker-w3-0f42cb-566b854747-xhdwq\" (UID: \"17ff1c66-c309-45da-9843-b37262d288d6\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-0f42cb-566b854747-xhdwq"
Oct 30 21:54:16.724832 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:16.724759    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vfqj4\" (UniqueName: \"kubernetes.io/projected/17ff1c66-c309-45da-9843-b37262d288d6-kube-api-access-vfqj4\") pod \"payload-processing-e2e-worker-w3-0f42cb-566b854747-xhdwq\" (UID: \"17ff1c66-c309-45da-9843-b37262d288d6\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-0f42cb-566b854747-xhdwq"
Oct 30 21:54:16.790969 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:16.790934    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w3-0f42cb-566b854747-xhdwq"
Oct 30 21:54:17.219700 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:17.219668    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-0f42cb-566b854747-xhdwq"]
Oct 30 21:54:17.220787 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:54:17.220755    4473 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17ff1c66_c309_45da_9843_b37262d288d6.slice/crio-8ad9d83d64fb3954c41ed8a89bf61a36a33cd3e3baa2d1b748f0157da6b7d713 WatchSource:0}: Error finding container 8ad9d83d64fb3954c41ed8a89bf61a36a33cd3e3baa2d1b748f0157da6b7d713: Status 404 returned error can't find the container with id 8ad9d83d64fb3954c41ed8a89bf61a36a33cd3e3baa2d1b748f0157da6b7d713
Oct 30 21:54:17.568504 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:17.568411    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w3-0f42cb-566b854747-xhdwq" event={"ID":"17ff1c66-c309-45da-9843-b37262d288d6","Type":"ContainerStarted","Data":"c47c4349be6968d4cac79d5fd1241de583baae04f584f112f57c91281fca9054"}
Oct 30 21:54:17.568504 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:17.568463    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w3-0f42cb-566b854747-xhdwq" event={"ID":"17ff1c66-c309-45da-9843-b37262d288d6","Type":"ContainerStarted","Data":"8ad9d83d64fb3954c41ed8a89bf61a36a33cd3e3baa2d1b748f0157da6b7d713"}
Oct 30 21:54:17.568504 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:17.568493    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w3-0f42cb-566b854747-xhdwq"
Oct 30 21:54:17.691285 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:17.691235    4473 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w3-0f42cb-566b854747-xhdwq" podStartSLOduration=1.691220241 podStartE2EDuration="1.691220241s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:54:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:54:17.689240657 +0000 UTC m=+703.462609157" watchObservedRunningTime="2026-10-30 21:54:17.691220241 +0000 UTC m=+703.464588764"
Oct 30 21:54:18.507580 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:18.507549    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-9f8aea-df9ff57d7-k7mlk"
Oct 30 21:54:18.866966 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:18.866937    4473 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-dde1e8-59fbfbdb67-vdb8w"]
Oct 30 21:54:18.870599 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:18.870584    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-dde1e8-59fbfbdb67-vdb8w"
Oct 30 21:54:18.941945 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:18.934267    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w0-dde1e8-dockercfg-jpzfl\""
Oct 30 21:54:18.941945 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:18.934375    4473 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w0-dde1e8\""
Oct 30 21:54:18.941945 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:18.935278    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-dde1e8-59fbfbdb67-vdb8w"]
Oct 30 21:54:18.999048 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:18.998957    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-njdz2\" (UniqueName: \"kubernetes.io/projected/40fb21f2-c4df-4bfd-b6e9-4198045570be-kube-api-access-njdz2\") pod \"payload-pre-processing-e2e-worker-w0-dde1e8-59fbfbdb67-vdb8w\" (UID: \"40fb21f2-c4df-4bfd-b6e9-4198045570be\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-dde1e8-59fbfbdb67-vdb8w"
Oct 30 21:54:18.999226 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:18.999155    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/40fb21f2-c4df-4bfd-b6e9-4198045570be-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-dde1e8-59fbfbdb67-vdb8w\" (UID: \"40fb21f2-c4df-4bfd-b6e9-4198045570be\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-dde1e8-59fbfbdb67-vdb8w"
Oct 30 21:54:19.023464 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:19.023429    4473 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-dde1e8-9575db797-2whh8"]
Oct 30 21:54:19.031949 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:19.027302    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-dde1e8-9575db797-2whh8"
Oct 30 21:54:19.087908 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:19.085526    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-dde1e8-9575db797-2whh8"]
Oct 30 21:54:19.100127 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:19.100097    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/35fcc100-bc0e-4c7a-9e34-aa06b86f288a-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-dde1e8-9575db797-2whh8\" (UID: \"35fcc100-bc0e-4c7a-9e34-aa06b86f288a\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-dde1e8-9575db797-2whh8"
Oct 30 21:54:19.100310 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:19.100251    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/40fb21f2-c4df-4bfd-b6e9-4198045570be-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-dde1e8-59fbfbdb67-vdb8w\" (UID: \"40fb21f2-c4df-4bfd-b6e9-4198045570be\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-dde1e8-59fbfbdb67-vdb8w"
Oct 30 21:54:19.100375 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:19.100332    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-njdz2\" (UniqueName: \"kubernetes.io/projected/40fb21f2-c4df-4bfd-b6e9-4198045570be-kube-api-access-njdz2\") pod \"payload-pre-processing-e2e-worker-w0-dde1e8-59fbfbdb67-vdb8w\" (UID: \"40fb21f2-c4df-4bfd-b6e9-4198045570be\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-dde1e8-59fbfbdb67-vdb8w"
Oct 30 21:54:19.100426 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:19.100380    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jcv2d\" (UniqueName: \"kubernetes.io/projected/35fcc100-bc0e-4c7a-9e34-aa06b86f288a-kube-api-access-jcv2d\") pod \"payload-processing-e2e-worker-w0-dde1e8-9575db797-2whh8\" (UID: \"35fcc100-bc0e-4c7a-9e34-aa06b86f288a\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-dde1e8-9575db797-2whh8"
Oct 30 21:54:19.100966 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:19.100912    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/40fb21f2-c4df-4bfd-b6e9-4198045570be-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-dde1e8-59fbfbdb67-vdb8w\" (UID: \"40fb21f2-c4df-4bfd-b6e9-4198045570be\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-dde1e8-59fbfbdb67-vdb8w"
Oct 30 21:54:19.162354 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:19.162280    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-njdz2\" (UniqueName: \"kubernetes.io/projected/40fb21f2-c4df-4bfd-b6e9-4198045570be-kube-api-access-njdz2\") pod \"payload-pre-processing-e2e-worker-w0-dde1e8-59fbfbdb67-vdb8w\" (UID: \"40fb21f2-c4df-4bfd-b6e9-4198045570be\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-dde1e8-59fbfbdb67-vdb8w"
Oct 30 21:54:19.185655 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:19.185098    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-dde1e8-59fbfbdb67-vdb8w"
Oct 30 21:54:19.201615 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:19.201580    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/35fcc100-bc0e-4c7a-9e34-aa06b86f288a-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-dde1e8-9575db797-2whh8\" (UID: \"35fcc100-bc0e-4c7a-9e34-aa06b86f288a\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-dde1e8-9575db797-2whh8"
Oct 30 21:54:19.202013 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:19.201986    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jcv2d\" (UniqueName: \"kubernetes.io/projected/35fcc100-bc0e-4c7a-9e34-aa06b86f288a-kube-api-access-jcv2d\") pod \"payload-processing-e2e-worker-w0-dde1e8-9575db797-2whh8\" (UID: \"35fcc100-bc0e-4c7a-9e34-aa06b86f288a\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-dde1e8-9575db797-2whh8"
Oct 30 21:54:19.202612 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:19.202588    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/35fcc100-bc0e-4c7a-9e34-aa06b86f288a-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-dde1e8-9575db797-2whh8\" (UID: \"35fcc100-bc0e-4c7a-9e34-aa06b86f288a\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-dde1e8-9575db797-2whh8"
Oct 30 21:54:19.262269 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:19.262236    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jcv2d\" (UniqueName: \"kubernetes.io/projected/35fcc100-bc0e-4c7a-9e34-aa06b86f288a-kube-api-access-jcv2d\") pod \"payload-processing-e2e-worker-w0-dde1e8-9575db797-2whh8\" (UID: \"35fcc100-bc0e-4c7a-9e34-aa06b86f288a\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-dde1e8-9575db797-2whh8"
Oct 30 21:54:19.371246 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:19.371205    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-dde1e8-9575db797-2whh8"
Oct 30 21:54:19.556710 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:19.556090    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-d516c4-687f696c99-xjfqk"
Oct 30 21:54:19.599805 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:19.599779    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-dde1e8-59fbfbdb67-vdb8w"]
Oct 30 21:54:19.600765 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:54:19.600737    4473 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40fb21f2_c4df_4bfd_b6e9_4198045570be.slice/crio-0687e3190b109823764c9a2f0aec2e75ccb748c6c80c96afb1b8e57494695494 WatchSource:0}: Error finding container 0687e3190b109823764c9a2f0aec2e75ccb748c6c80c96afb1b8e57494695494: Status 404 returned error can't find the container with id 0687e3190b109823764c9a2f0aec2e75ccb748c6c80c96afb1b8e57494695494
Oct 30 21:54:19.785765 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:19.785736    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-dde1e8-9575db797-2whh8"]
Oct 30 21:54:19.787113 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:54:19.787088    4473 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod35fcc100_bc0e_4c7a_9e34_aa06b86f288a.slice/crio-8de58918b86455086f1fcb0c2d5b912d24b67e1c6c42104dcef4fc8889e25a5c WatchSource:0}: Error finding container 8de58918b86455086f1fcb0c2d5b912d24b67e1c6c42104dcef4fc8889e25a5c: Status 404 returned error can't find the container with id 8de58918b86455086f1fcb0c2d5b912d24b67e1c6c42104dcef4fc8889e25a5c
Oct 30 21:54:20.325726 ip-10-0-140-18 kubenswrapper[4473]: E1030 21:54:20.325692    4473 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fncvg" containerName="extractor"
Oct 30 21:54:20.595439 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:20.595113    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-dde1e8-9575db797-2whh8" event={"ID":"35fcc100-bc0e-4c7a-9e34-aa06b86f288a","Type":"ContainerStarted","Data":"212ee3847d8144b97dc7b6de30a8a3ff313835f090a9da740d069666b6c9fd2b"}
Oct 30 21:54:20.595439 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:20.595183    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w0-dde1e8-9575db797-2whh8"
Oct 30 21:54:20.595439 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:20.595210    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-dde1e8-9575db797-2whh8" event={"ID":"35fcc100-bc0e-4c7a-9e34-aa06b86f288a","Type":"ContainerStarted","Data":"8de58918b86455086f1fcb0c2d5b912d24b67e1c6c42104dcef4fc8889e25a5c"}
Oct 30 21:54:20.596892 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:20.596862    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-dde1e8-59fbfbdb67-vdb8w" event={"ID":"40fb21f2-c4df-4bfd-b6e9-4198045570be","Type":"ContainerStarted","Data":"1a7b98080bc79bb81666c15e653c237883b2d9eb22ade815b772790a02231b1a"}
Oct 30 21:54:20.597059 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:20.596913    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-dde1e8-59fbfbdb67-vdb8w" event={"ID":"40fb21f2-c4df-4bfd-b6e9-4198045570be","Type":"ContainerStarted","Data":"0687e3190b109823764c9a2f0aec2e75ccb748c6c80c96afb1b8e57494695494"}
Oct 30 21:54:20.597059 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:20.596973    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-dde1e8-59fbfbdb67-vdb8w"
Oct 30 21:54:20.715997 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:20.715949    4473 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w0-dde1e8-9575db797-2whh8" podStartSLOduration=2.715932804 podStartE2EDuration="2.715932804s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:54:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:54:20.714755156 +0000 UTC m=+706.488123644" watchObservedRunningTime="2026-10-30 21:54:20.715932804 +0000 UTC m=+706.489301282"
Oct 30 21:54:20.841502 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:20.841438    4473 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-dde1e8-59fbfbdb67-vdb8w" podStartSLOduration=2.841418465 podStartE2EDuration="2.841418465s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:54:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:54:20.838503855 +0000 UTC m=+706.611872345" watchObservedRunningTime="2026-10-30 21:54:20.841418465 +0000 UTC m=+706.614786953"
Oct 30 21:54:23.450126 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:23.450073    4473 patch_prober.go:29] interesting pod/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-xct57 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:54:23.450636 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:23.450154    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-xct57" podUID="893a5438-3b54-4d77-8675-00260980c0f6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:54:24.617830 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:24.617799    4473 generic.go:356] "Generic (PLEG): container finished" podID="893a5438-3b54-4d77-8675-00260980c0f6" containerID="21ac28e226e4fd387318bd11bb97c8354829aaf45ba244118fbd6ac71dfb1b36" exitCode=0
Oct 30 21:54:24.618293 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:24.617858    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-xct57" event={"ID":"893a5438-3b54-4d77-8675-00260980c0f6","Type":"ContainerDied","Data":"21ac28e226e4fd387318bd11bb97c8354829aaf45ba244118fbd6ac71dfb1b36"}
Oct 30 21:54:24.618293 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:24.618290    4473 scope.go:122] "RemoveContainer" containerID="21ac28e226e4fd387318bd11bb97c8354829aaf45ba244118fbd6ac71dfb1b36"
Oct 30 21:54:24.619591 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:24.619564    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:54:24.619643 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:24.619632    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:54:24.619677 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:24.619665    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:54:25.642948 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:25.642055    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-xct57" event={"ID":"893a5438-3b54-4d77-8675-00260980c0f6","Type":"ContainerStarted","Data":"1e44b133e1aebad98c1030d44c6e8dc5f3788dba7b582f7f6cd220bcd712a9cd"}
Oct 30 21:54:26.378051 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:26.378016    4473 patch_prober.go:29] interesting pod/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-xct57 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body=
Oct 30 21:54:26.378340 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:26.378310    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-xct57" podUID="893a5438-3b54-4d77-8675-00260980c0f6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused"
Oct 30 21:54:26.382695 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:26.382661    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-xct57"
Oct 30 21:54:27.372947 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:27.372353    4473 patch_prober.go:29] interesting pod/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-xct57 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body=
Oct 30 21:54:27.372947 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:27.372422    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-xct57" podUID="893a5438-3b54-4d77-8675-00260980c0f6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused"
Oct 30 21:54:28.371960 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:28.371903    4473 patch_prober.go:29] interesting pod/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-xct57 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body=
Oct 30 21:54:28.372175 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:28.371984    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-xct57" podUID="893a5438-3b54-4d77-8675-00260980c0f6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused"
Oct 30 21:54:28.589098 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:28.589065    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w3-0f42cb-566b854747-xhdwq"
Oct 30 21:54:29.372224 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:29.372183    4473 patch_prober.go:29] interesting pod/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-xct57 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body=
Oct 30 21:54:29.372413 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:29.372297    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-xct57" podUID="893a5438-3b54-4d77-8675-00260980c0f6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused"
Oct 30 21:54:30.372565 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:30.372526    4473 patch_prober.go:29] interesting pod/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-xct57 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body=
Oct 30 21:54:30.373103 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:30.372582    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-xct57" podUID="893a5438-3b54-4d77-8675-00260980c0f6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused"
Oct 30 21:54:31.376422 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:31.376391    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-xct57"
Oct 30 21:54:31.376954 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:31.376791    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-xct57"
Oct 30 21:54:31.377711 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:31.377691    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-xct57"
Oct 30 21:54:31.604856 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:31.604824    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w0-dde1e8-9575db797-2whh8"
Oct 30 21:54:31.605050 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:31.604894    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-dde1e8-59fbfbdb67-vdb8w"
Oct 30 21:54:32.713271 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:32.713235    4473 generic.go:356] "Generic (PLEG): container finished" podID="335708bc-083d-4ce1-9158-cc61dcac93da" containerID="f8e22d2edf99ab821c2b98a87d18effbb6bb9b17a0e4ac6ec2693fd3826b85dd" exitCode=0
Oct 30 21:54:32.713706 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:32.713311    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-bs4lr" event={"ID":"335708bc-083d-4ce1-9158-cc61dcac93da","Type":"ContainerDied","Data":"f8e22d2edf99ab821c2b98a87d18effbb6bb9b17a0e4ac6ec2693fd3826b85dd"}
Oct 30 21:54:32.713993 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:32.713976    4473 scope.go:122] "RemoveContainer" containerID="f8e22d2edf99ab821c2b98a87d18effbb6bb9b17a0e4ac6ec2693fd3826b85dd"
Oct 30 21:54:32.715218 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:32.715188    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:54:32.715300 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:32.715255    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:54:32.715300 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:32.715288    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:54:33.720736 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:33.720703    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-bs4lr" event={"ID":"335708bc-083d-4ce1-9158-cc61dcac93da","Type":"ContainerStarted","Data":"3fe63c4f9af714c3502e0939e441ec7caaa4ce6fec3f1467b36f57facc47ddf8"}
Oct 30 21:54:34.058965 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:34.058842    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-bs4lr"
Oct 30 21:54:34.060316 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:34.060287    4473 patch_prober.go:29] interesting pod/e2e-worker-w1-d516c4-openshift-default-6bd584745-bs4lr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body=
Oct 30 21:54:34.060445 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:34.060339    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-bs4lr" podUID="335708bc-083d-4ce1-9158-cc61dcac93da" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Oct 30 21:54:35.059062 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:35.059017    4473 patch_prober.go:29] interesting pod/e2e-worker-w1-d516c4-openshift-default-6bd584745-bs4lr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body=
Oct 30 21:54:35.059633 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:35.059084    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-bs4lr" podUID="335708bc-083d-4ce1-9158-cc61dcac93da" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Oct 30 21:54:36.058764 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:36.058723    4473 patch_prober.go:29] interesting pod/e2e-worker-w1-d516c4-openshift-default-6bd584745-bs4lr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body=
Oct 30 21:54:36.058982 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:36.058782    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-bs4lr" podUID="335708bc-083d-4ce1-9158-cc61dcac93da" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Oct 30 21:54:37.058240 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:37.058203    4473 patch_prober.go:29] interesting pod/e2e-worker-w1-d516c4-openshift-default-6bd584745-bs4lr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body=
Oct 30 21:54:37.058684 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:37.058267    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-bs4lr" podUID="335708bc-083d-4ce1-9158-cc61dcac93da" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Oct 30 21:54:38.067558 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:38.067499    4473 patch_prober.go:29] interesting pod/e2e-worker-w1-d516c4-openshift-default-6bd584745-bs4lr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body=
Oct 30 21:54:38.068366 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:38.068318    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-bs4lr" podUID="335708bc-083d-4ce1-9158-cc61dcac93da" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Oct 30 21:54:38.401848 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:38.401810    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-bs4lr"
Oct 30 21:54:39.074619 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:39.074572    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-bs4lr"
Oct 30 21:54:39.075941 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:39.075900    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-bs4lr"
Oct 30 21:54:43.294699 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:43.294663    4473 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-dde1e8-7d47cf4786-8d75q"]
Oct 30 21:54:43.298093 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:43.298073    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-dde1e8-7d47cf4786-8d75q"
Oct 30 21:54:43.388466 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:43.388417    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bz2lh\" (UniqueName: \"kubernetes.io/projected/b372cc09-4720-4855-9c3a-0e8e7b8beb70-kube-api-access-bz2lh\") pod \"maas-api-e2e-worker-w0-dde1e8-7d47cf4786-8d75q\" (UID: \"b372cc09-4720-4855-9c3a-0e8e7b8beb70\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-dde1e8-7d47cf4786-8d75q"
Oct 30 21:54:43.388466 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:43.388461    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b372cc09-4720-4855-9c3a-0e8e7b8beb70-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-dde1e8-7d47cf4786-8d75q\" (UID: \"b372cc09-4720-4855-9c3a-0e8e7b8beb70\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-dde1e8-7d47cf4786-8d75q"
Oct 30 21:54:43.388752 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:43.388550    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b372cc09-4720-4855-9c3a-0e8e7b8beb70-maas-api-tls\") pod \"maas-api-e2e-worker-w0-dde1e8-7d47cf4786-8d75q\" (UID: \"b372cc09-4720-4855-9c3a-0e8e7b8beb70\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-dde1e8-7d47cf4786-8d75q"
Oct 30 21:54:43.388752 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:43.388585    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b372cc09-4720-4855-9c3a-0e8e7b8beb70-metrics-certs\") pod \"maas-api-e2e-worker-w0-dde1e8-7d47cf4786-8d75q\" (UID: \"b372cc09-4720-4855-9c3a-0e8e7b8beb70\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-dde1e8-7d47cf4786-8d75q"
Oct 30 21:54:43.404271 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:43.404230    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w0-dde1e8-serving-cert\""
Oct 30 21:54:43.406745 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:43.406708    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-dde1e8-7d47cf4786-8d75q"]
Oct 30 21:54:43.489433 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:43.489396    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b372cc09-4720-4855-9c3a-0e8e7b8beb70-maas-api-tls\") pod \"maas-api-e2e-worker-w0-dde1e8-7d47cf4786-8d75q\" (UID: \"b372cc09-4720-4855-9c3a-0e8e7b8beb70\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-dde1e8-7d47cf4786-8d75q"
Oct 30 21:54:43.489433 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:43.489429    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b372cc09-4720-4855-9c3a-0e8e7b8beb70-metrics-certs\") pod \"maas-api-e2e-worker-w0-dde1e8-7d47cf4786-8d75q\" (UID: \"b372cc09-4720-4855-9c3a-0e8e7b8beb70\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-dde1e8-7d47cf4786-8d75q"
Oct 30 21:54:43.489713 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:43.489547    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bz2lh\" (UniqueName: \"kubernetes.io/projected/b372cc09-4720-4855-9c3a-0e8e7b8beb70-kube-api-access-bz2lh\") pod \"maas-api-e2e-worker-w0-dde1e8-7d47cf4786-8d75q\" (UID: \"b372cc09-4720-4855-9c3a-0e8e7b8beb70\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-dde1e8-7d47cf4786-8d75q"
Oct 30 21:54:43.489713 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:43.489566    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b372cc09-4720-4855-9c3a-0e8e7b8beb70-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-dde1e8-7d47cf4786-8d75q\" (UID: \"b372cc09-4720-4855-9c3a-0e8e7b8beb70\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-dde1e8-7d47cf4786-8d75q"
Oct 30 21:54:43.489713 ip-10-0-140-18 kubenswrapper[4473]: E1030 21:54:43.489577    4473 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w0-dde1e8-serving-cert: secret "maas-api-e2e-worker-w0-dde1e8-serving-cert" not found
Oct 30 21:54:43.489713 ip-10-0-140-18 kubenswrapper[4473]: E1030 21:54:43.489664    4473 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b372cc09-4720-4855-9c3a-0e8e7b8beb70-maas-api-tls podName:b372cc09-4720-4855-9c3a-0e8e7b8beb70 nodeName:}" failed. No retries permitted until 2026-10-30 21:54:43.989640226 +0000 UTC m=+729.763008694 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/b372cc09-4720-4855-9c3a-0e8e7b8beb70-maas-api-tls") pod "maas-api-e2e-worker-w0-dde1e8-7d47cf4786-8d75q" (UID: "b372cc09-4720-4855-9c3a-0e8e7b8beb70") : secret "maas-api-e2e-worker-w0-dde1e8-serving-cert" not found
Oct 30 21:54:43.490206 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:43.490184    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b372cc09-4720-4855-9c3a-0e8e7b8beb70-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-dde1e8-7d47cf4786-8d75q\" (UID: \"b372cc09-4720-4855-9c3a-0e8e7b8beb70\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-dde1e8-7d47cf4786-8d75q"
Oct 30 21:54:43.492301 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:43.492278    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b372cc09-4720-4855-9c3a-0e8e7b8beb70-metrics-certs\") pod \"maas-api-e2e-worker-w0-dde1e8-7d47cf4786-8d75q\" (UID: \"b372cc09-4720-4855-9c3a-0e8e7b8beb70\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-dde1e8-7d47cf4786-8d75q"
Oct 30 21:54:43.586327 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:43.568767    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bz2lh\" (UniqueName: \"kubernetes.io/projected/b372cc09-4720-4855-9c3a-0e8e7b8beb70-kube-api-access-bz2lh\") pod \"maas-api-e2e-worker-w0-dde1e8-7d47cf4786-8d75q\" (UID: \"b372cc09-4720-4855-9c3a-0e8e7b8beb70\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-dde1e8-7d47cf4786-8d75q"
Oct 30 21:54:43.994515 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:43.994481    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b372cc09-4720-4855-9c3a-0e8e7b8beb70-maas-api-tls\") pod \"maas-api-e2e-worker-w0-dde1e8-7d47cf4786-8d75q\" (UID: \"b372cc09-4720-4855-9c3a-0e8e7b8beb70\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-dde1e8-7d47cf4786-8d75q"
Oct 30 21:54:43.997123 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:43.997096    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b372cc09-4720-4855-9c3a-0e8e7b8beb70-maas-api-tls\") pod \"maas-api-e2e-worker-w0-dde1e8-7d47cf4786-8d75q\" (UID: \"b372cc09-4720-4855-9c3a-0e8e7b8beb70\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-dde1e8-7d47cf4786-8d75q"
Oct 30 21:54:44.210396 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:44.210356    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-dde1e8-7d47cf4786-8d75q"
Oct 30 21:54:44.589432 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:44.589404    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-dde1e8-7d47cf4786-8d75q"]
Oct 30 21:54:44.591132 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:54:44.591104    4473 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb372cc09_4720_4855_9c3a_0e8e7b8beb70.slice/crio-ada95f347f28865d75ad9c6b62f7f1ce7bf5e78eef56b25ac7c430db937ed45d WatchSource:0}: Error finding container ada95f347f28865d75ad9c6b62f7f1ce7bf5e78eef56b25ac7c430db937ed45d: Status 404 returned error can't find the container with id ada95f347f28865d75ad9c6b62f7f1ce7bf5e78eef56b25ac7c430db937ed45d
Oct 30 21:54:44.804245 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:44.804211    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-dde1e8-7d47cf4786-8d75q" event={"ID":"b372cc09-4720-4855-9c3a-0e8e7b8beb70","Type":"ContainerStarted","Data":"ada95f347f28865d75ad9c6b62f7f1ce7bf5e78eef56b25ac7c430db937ed45d"}
Oct 30 21:54:45.811622 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:45.811518    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-dde1e8-7d47cf4786-8d75q" event={"ID":"b372cc09-4720-4855-9c3a-0e8e7b8beb70","Type":"ContainerStarted","Data":"d37b59a9ced77e902cd29542a330609e06d4e6055e5fa7e9b482356aa3fdf4e0"}
Oct 30 21:54:45.811622 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:45.811608    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-dde1e8-7d47cf4786-8d75q"
Oct 30 21:54:45.928881 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:45.928827    4473 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-dde1e8-7d47cf4786-8d75q" podStartSLOduration=1.9885967610000002 podStartE2EDuration="2.928811722s" totalImagesPullingTime="940.214961ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:54:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:54:45.927050294 +0000 UTC m=+731.700418781" watchObservedRunningTime="2026-10-30 21:54:45.928811722 +0000 UTC m=+731.702180208"
Oct 30 21:54:47.824631 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:47.824601    4473 generic.go:356] "Generic (PLEG): container finished" podID="6560c1bc-b507-4a92-8586-23e0fcc5362a" containerID="66d5162ae5be3f6186045d73c5997ea750a278198e32e17fc0b5624548b96834" exitCode=0
Oct 30 21:54:47.825137 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:47.824697    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-qng2w" event={"ID":"6560c1bc-b507-4a92-8586-23e0fcc5362a","Type":"ContainerDied","Data":"66d5162ae5be3f6186045d73c5997ea750a278198e32e17fc0b5624548b96834"}
Oct 30 21:54:47.825187 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:47.825149    4473 scope.go:122] "RemoveContainer" containerID="66d5162ae5be3f6186045d73c5997ea750a278198e32e17fc0b5624548b96834"
Oct 30 21:54:47.826501 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:47.826472    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:54:47.826572 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:47.826540    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:54:47.826615 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:47.826573    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:54:48.855828 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:48.855788    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-qng2w" event={"ID":"6560c1bc-b507-4a92-8586-23e0fcc5362a","Type":"ContainerStarted","Data":"d26e70139b180333d21d0d8011cf38c246c5461712eb9266e0a20acae0653eae"}
Oct 30 21:54:49.741325 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:49.741292    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-qng2w"
Oct 30 21:54:49.741742 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:49.741719    4473 patch_prober.go:29] interesting pod/e2e-worker-w2-0c99b1-openshift-default-967855977-qng2w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body=
Oct 30 21:54:49.741903 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:49.741881    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-qng2w" podUID="6560c1bc-b507-4a92-8586-23e0fcc5362a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused"
Oct 30 21:54:50.324399 ip-10-0-140-18 kubenswrapper[4473]: E1030 21:54:50.324365    4473 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fncvg" containerName="extractor"
Oct 30 21:54:50.744948 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:50.741410    4473 patch_prober.go:29] interesting pod/e2e-worker-w2-0c99b1-openshift-default-967855977-qng2w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body=
Oct 30 21:54:50.744948 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:50.741480    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-qng2w" podUID="6560c1bc-b507-4a92-8586-23e0fcc5362a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused"
Oct 30 21:54:51.744947 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:51.742344    4473 patch_prober.go:29] interesting pod/e2e-worker-w2-0c99b1-openshift-default-967855977-qng2w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body=
Oct 30 21:54:51.744947 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:51.742404    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-qng2w" podUID="6560c1bc-b507-4a92-8586-23e0fcc5362a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused"
Oct 30 21:54:51.823482 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:51.823455    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-dde1e8-7d47cf4786-8d75q"
Oct 30 21:54:52.396667 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:52.396026    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-qng2w"
Oct 30 21:54:52.745441 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:52.745345    4473 patch_prober.go:29] interesting pod/e2e-worker-w2-0c99b1-openshift-default-967855977-qng2w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body=
Oct 30 21:54:52.746059 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:52.746023    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-qng2w" podUID="6560c1bc-b507-4a92-8586-23e0fcc5362a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused"
Oct 30 21:54:53.745944 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:53.745444    4473 patch_prober.go:29] interesting pod/e2e-worker-w2-0c99b1-openshift-default-967855977-qng2w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body=
Oct 30 21:54:53.745944 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:53.745522    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-qng2w" podUID="6560c1bc-b507-4a92-8586-23e0fcc5362a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused"
Oct 30 21:54:54.754360 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:54.754331    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-qng2w"
Oct 30 21:54:54.755505 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:54.755484    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-qng2w"
Oct 30 21:54:58.940154 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:58.940121    4473 generic.go:356] "Generic (PLEG): container finished" podID="0f0fbc43-d98b-4b7c-872b-7978f76650ef" containerID="3b0fe4884f0af7c72480e3c91905a0dd01fe4f95e9c9ba8c7665ad3c1f926e4c" exitCode=0
Oct 30 21:54:58.940614 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:58.940194    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-j5tqr" event={"ID":"0f0fbc43-d98b-4b7c-872b-7978f76650ef","Type":"ContainerDied","Data":"3b0fe4884f0af7c72480e3c91905a0dd01fe4f95e9c9ba8c7665ad3c1f926e4c"}
Oct 30 21:54:58.940614 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:58.940591    4473 scope.go:122] "RemoveContainer" containerID="3b0fe4884f0af7c72480e3c91905a0dd01fe4f95e9c9ba8c7665ad3c1f926e4c"
Oct 30 21:54:58.942120 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:58.942087    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:54:58.942195 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:58.942165    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:54:58.942231 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:58.942204    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:54:59.960314 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:54:59.960275    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-j5tqr" event={"ID":"0f0fbc43-d98b-4b7c-872b-7978f76650ef","Type":"ContainerStarted","Data":"93f6fe2bc6310bf844bba8d8e01162b501bb6d2f275f229926ca00321452255c"}
Oct 30 21:55:00.443826 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:55:00.443789    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-j5tqr"
Oct 30 21:55:00.444206 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:55:00.444183    4473 patch_prober.go:29] interesting pod/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-j5tqr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body=
Oct 30 21:55:00.444360 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:55:00.444335    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-j5tqr" podUID="0f0fbc43-d98b-4b7c-872b-7978f76650ef" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused"
Oct 30 21:55:01.439429 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:55:01.439383    4473 patch_prober.go:29] interesting pod/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-j5tqr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body=
Oct 30 21:55:01.440090 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:55:01.440054    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-j5tqr" podUID="0f0fbc43-d98b-4b7c-872b-7978f76650ef" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused"
Oct 30 21:55:02.450151 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:55:02.450106    4473 patch_prober.go:29] interesting pod/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-j5tqr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body=
Oct 30 21:55:02.450652 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:55:02.450170    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-j5tqr" podUID="0f0fbc43-d98b-4b7c-872b-7978f76650ef" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused"
Oct 30 21:55:03.439947 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:55:03.436335    4473 patch_prober.go:29] interesting pod/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-j5tqr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body=
Oct 30 21:55:03.439947 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:55:03.436405    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-j5tqr" podUID="0f0fbc43-d98b-4b7c-872b-7978f76650ef" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused"
Oct 30 21:55:04.439818 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:55:04.439782    4473 patch_prober.go:29] interesting pod/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-j5tqr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body=
Oct 30 21:55:04.440477 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:55:04.440441    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-j5tqr" podUID="0f0fbc43-d98b-4b7c-872b-7978f76650ef" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused"
Oct 30 21:55:05.435876 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:55:05.435845    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-j5tqr"
Oct 30 21:55:05.436164 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:55:05.436145    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-j5tqr"
Oct 30 21:55:05.437018 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:55:05.437000    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-j5tqr"
Oct 30 21:55:20.325437 ip-10-0-140-18 kubenswrapper[4473]: E1030 21:55:20.325347    4473 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fncvg" containerName="extractor"
Oct 30 21:55:50.324595 ip-10-0-140-18 kubenswrapper[4473]: E1030 21:55:50.324546    4473 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fncvg" containerName="extractor"
Oct 30 21:56:05.927138 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:56:05.927096    4473 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-mg8sv"]
Oct 30 21:56:05.930345 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:56:05.930319    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-mg8sv"
Oct 30 21:56:05.984468 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:56:05.984439    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w4-b0c55f-openshift-default-dockercfg-gqvhz\""
Oct 30 21:56:05.992680 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:56:05.992645    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/117cd302-f5bc-4c57-9900-1b6ddb8c5e25-workload-certs\") pod \"e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-mg8sv\" (UID: \"117cd302-f5bc-4c57-9900-1b6ddb8c5e25\") " pod="openshift-ingress/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-mg8sv"
Oct 30 21:56:05.992816 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:56:05.992703    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-szrl4\" (UniqueName: \"kubernetes.io/projected/117cd302-f5bc-4c57-9900-1b6ddb8c5e25-kube-api-access-szrl4\") pod \"e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-mg8sv\" (UID: \"117cd302-f5bc-4c57-9900-1b6ddb8c5e25\") " pod="openshift-ingress/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-mg8sv"
Oct 30 21:56:05.992816 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:56:05.992772    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/117cd302-f5bc-4c57-9900-1b6ddb8c5e25-istio-envoy\") pod \"e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-mg8sv\" (UID: \"117cd302-f5bc-4c57-9900-1b6ddb8c5e25\") " pod="openshift-ingress/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-mg8sv"
Oct 30 21:56:05.992900 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:56:05.992819    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/117cd302-f5bc-4c57-9900-1b6ddb8c5e25-workload-socket\") pod \"e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-mg8sv\" (UID: \"117cd302-f5bc-4c57-9900-1b6ddb8c5e25\") " pod="openshift-ingress/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-mg8sv"
Oct 30 21:56:05.992900 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:56:05.992840    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/117cd302-f5bc-4c57-9900-1b6ddb8c5e25-istio-podinfo\") pod \"e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-mg8sv\" (UID: \"117cd302-f5bc-4c57-9900-1b6ddb8c5e25\") " pod="openshift-ingress/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-mg8sv"
Oct 30 21:56:05.993005 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:56:05.992976    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/117cd302-f5bc-4c57-9900-1b6ddb8c5e25-istio-token\") pod \"e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-mg8sv\" (UID: \"117cd302-f5bc-4c57-9900-1b6ddb8c5e25\") " pod="openshift-ingress/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-mg8sv"
Oct 30 21:56:05.993049 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:56:05.993006    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/117cd302-f5bc-4c57-9900-1b6ddb8c5e25-istio-data\") pod \"e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-mg8sv\" (UID: \"117cd302-f5bc-4c57-9900-1b6ddb8c5e25\") " pod="openshift-ingress/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-mg8sv"
Oct 30 21:56:05.993049 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:56:05.993028    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/117cd302-f5bc-4c57-9900-1b6ddb8c5e25-credential-socket\") pod \"e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-mg8sv\" (UID: \"117cd302-f5bc-4c57-9900-1b6ddb8c5e25\") " pod="openshift-ingress/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-mg8sv"
Oct 30 21:56:05.993138 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:56:05.993124    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/117cd302-f5bc-4c57-9900-1b6ddb8c5e25-istiod-ca-cert\") pod \"e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-mg8sv\" (UID: \"117cd302-f5bc-4c57-9900-1b6ddb8c5e25\") " pod="openshift-ingress/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-mg8sv"
Oct 30 21:56:06.042407 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:56:06.042369    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-mg8sv"]
Oct 30 21:56:06.093758 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:56:06.093712    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/117cd302-f5bc-4c57-9900-1b6ddb8c5e25-istio-data\") pod \"e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-mg8sv\" (UID: \"117cd302-f5bc-4c57-9900-1b6ddb8c5e25\") " pod="openshift-ingress/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-mg8sv"
Oct 30 21:56:06.093758 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:56:06.093759    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/117cd302-f5bc-4c57-9900-1b6ddb8c5e25-credential-socket\") pod \"e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-mg8sv\" (UID: \"117cd302-f5bc-4c57-9900-1b6ddb8c5e25\") " pod="openshift-ingress/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-mg8sv"
Oct 30 21:56:06.094083 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:56:06.093787    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/117cd302-f5bc-4c57-9900-1b6ddb8c5e25-istiod-ca-cert\") pod \"e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-mg8sv\" (UID: \"117cd302-f5bc-4c57-9900-1b6ddb8c5e25\") " pod="openshift-ingress/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-mg8sv"
Oct 30 21:56:06.094083 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:56:06.093941    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/117cd302-f5bc-4c57-9900-1b6ddb8c5e25-workload-certs\") pod \"e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-mg8sv\" (UID: \"117cd302-f5bc-4c57-9900-1b6ddb8c5e25\") " pod="openshift-ingress/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-mg8sv"
Oct 30 21:56:06.094083 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:56:06.094048    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-szrl4\" (UniqueName: \"kubernetes.io/projected/117cd302-f5bc-4c57-9900-1b6ddb8c5e25-kube-api-access-szrl4\") pod \"e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-mg8sv\" (UID: \"117cd302-f5bc-4c57-9900-1b6ddb8c5e25\") " pod="openshift-ingress/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-mg8sv"
Oct 30 21:56:06.094244 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:56:06.094085    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/117cd302-f5bc-4c57-9900-1b6ddb8c5e25-istio-envoy\") pod \"e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-mg8sv\" (UID: \"117cd302-f5bc-4c57-9900-1b6ddb8c5e25\") " pod="openshift-ingress/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-mg8sv"
Oct 30 21:56:06.094244 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:56:06.094150    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/117cd302-f5bc-4c57-9900-1b6ddb8c5e25-workload-socket\") pod \"e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-mg8sv\" (UID: \"117cd302-f5bc-4c57-9900-1b6ddb8c5e25\") " pod="openshift-ingress/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-mg8sv"
Oct 30 21:56:06.094244 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:56:06.094184    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/117cd302-f5bc-4c57-9900-1b6ddb8c5e25-istio-podinfo\") pod \"e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-mg8sv\" (UID: \"117cd302-f5bc-4c57-9900-1b6ddb8c5e25\") " pod="openshift-ingress/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-mg8sv"
Oct 30 21:56:06.094244 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:56:06.094214    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/117cd302-f5bc-4c57-9900-1b6ddb8c5e25-istio-data\") pod \"e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-mg8sv\" (UID: \"117cd302-f5bc-4c57-9900-1b6ddb8c5e25\") " pod="openshift-ingress/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-mg8sv"
Oct 30 21:56:06.094430 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:56:06.094260    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/117cd302-f5bc-4c57-9900-1b6ddb8c5e25-workload-certs\") pod \"e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-mg8sv\" (UID: \"117cd302-f5bc-4c57-9900-1b6ddb8c5e25\") " pod="openshift-ingress/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-mg8sv"
Oct 30 21:56:06.094430 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:56:06.094286    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/117cd302-f5bc-4c57-9900-1b6ddb8c5e25-istio-token\") pod \"e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-mg8sv\" (UID: \"117cd302-f5bc-4c57-9900-1b6ddb8c5e25\") " pod="openshift-ingress/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-mg8sv"
Oct 30 21:56:06.094582 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:56:06.094553    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/117cd302-f5bc-4c57-9900-1b6ddb8c5e25-workload-socket\") pod \"e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-mg8sv\" (UID: \"117cd302-f5bc-4c57-9900-1b6ddb8c5e25\") " pod="openshift-ingress/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-mg8sv"
Oct 30 21:56:06.094656 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:56:06.094585    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/117cd302-f5bc-4c57-9900-1b6ddb8c5e25-credential-socket\") pod \"e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-mg8sv\" (UID: \"117cd302-f5bc-4c57-9900-1b6ddb8c5e25\") " pod="openshift-ingress/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-mg8sv"
Oct 30 21:56:06.094656 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:56:06.094628    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/117cd302-f5bc-4c57-9900-1b6ddb8c5e25-istiod-ca-cert\") pod \"e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-mg8sv\" (UID: \"117cd302-f5bc-4c57-9900-1b6ddb8c5e25\") " pod="openshift-ingress/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-mg8sv"
Oct 30 21:56:06.096607 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:56:06.096582    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/117cd302-f5bc-4c57-9900-1b6ddb8c5e25-istio-envoy\") pod \"e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-mg8sv\" (UID: \"117cd302-f5bc-4c57-9900-1b6ddb8c5e25\") " pod="openshift-ingress/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-mg8sv"
Oct 30 21:56:06.096713 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:56:06.096678    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/117cd302-f5bc-4c57-9900-1b6ddb8c5e25-istio-podinfo\") pod \"e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-mg8sv\" (UID: \"117cd302-f5bc-4c57-9900-1b6ddb8c5e25\") " pod="openshift-ingress/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-mg8sv"
Oct 30 21:56:06.152551 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:56:06.152514    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/117cd302-f5bc-4c57-9900-1b6ddb8c5e25-istio-token\") pod \"e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-mg8sv\" (UID: \"117cd302-f5bc-4c57-9900-1b6ddb8c5e25\") " pod="openshift-ingress/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-mg8sv"
Oct 30 21:56:06.152731 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:56:06.152618    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-szrl4\" (UniqueName: \"kubernetes.io/projected/117cd302-f5bc-4c57-9900-1b6ddb8c5e25-kube-api-access-szrl4\") pod \"e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-mg8sv\" (UID: \"117cd302-f5bc-4c57-9900-1b6ddb8c5e25\") " pod="openshift-ingress/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-mg8sv"
Oct 30 21:56:06.243765 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:56:06.243654    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-mg8sv"
Oct 30 21:56:06.670106 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:56:06.670077    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-mg8sv"]
Oct 30 21:56:06.672159 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:56:06.672123    4473 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod117cd302_f5bc_4c57_9900_1b6ddb8c5e25.slice/crio-ff51301e3d8951109a7ec2dcbf410000acafe044654bfa7253e6bd11ca8a5b64 WatchSource:0}: Error finding container ff51301e3d8951109a7ec2dcbf410000acafe044654bfa7253e6bd11ca8a5b64: Status 404 returned error can't find the container with id ff51301e3d8951109a7ec2dcbf410000acafe044654bfa7253e6bd11ca8a5b64
Oct 30 21:56:06.674710 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:56:06.674671    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:56:06.674812 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:56:06.674744    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:56:06.674812 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:56:06.674795    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:56:07.357693 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:56:07.357652    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-mg8sv" event={"ID":"117cd302-f5bc-4c57-9900-1b6ddb8c5e25","Type":"ContainerStarted","Data":"6b0dac1643a2721da8037f583e9119f35875e488b01803a244c4fc71eb136065"}
Oct 30 21:56:07.358147 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:56:07.357766    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-mg8sv" event={"ID":"117cd302-f5bc-4c57-9900-1b6ddb8c5e25","Type":"ContainerStarted","Data":"ff51301e3d8951109a7ec2dcbf410000acafe044654bfa7253e6bd11ca8a5b64"}
Oct 30 21:56:07.483725 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:56:07.483667    4473 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-mg8sv" podStartSLOduration=2.483651431 podStartE2EDuration="2.483651431s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:56:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:56:07.480790657 +0000 UTC m=+813.254159135" watchObservedRunningTime="2026-10-30 21:56:07.483651431 +0000 UTC m=+813.257019918"
Oct 30 21:56:08.249543 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:56:08.249505    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-mg8sv"
Oct 30 21:56:08.249989 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:56:08.249879    4473 patch_prober.go:29] interesting pod/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-mg8sv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body=
Oct 30 21:56:08.250174 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:56:08.250150    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-mg8sv" podUID="117cd302-f5bc-4c57-9900-1b6ddb8c5e25" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused"
Oct 30 21:56:09.246427 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:56:09.246383    4473 patch_prober.go:29] interesting pod/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-mg8sv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body=
Oct 30 21:56:09.247118 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:56:09.247075    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-mg8sv" podUID="117cd302-f5bc-4c57-9900-1b6ddb8c5e25" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused"
Oct 30 21:56:10.248333 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:56:10.248295    4473 patch_prober.go:29] interesting pod/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-mg8sv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body=
Oct 30 21:56:10.248999 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:56:10.248957    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-mg8sv" podUID="117cd302-f5bc-4c57-9900-1b6ddb8c5e25" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused"
Oct 30 21:56:11.250564 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:56:11.250526    4473 patch_prober.go:29] interesting pod/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-mg8sv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body=
Oct 30 21:56:11.251267 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:56:11.251228    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-mg8sv" podUID="117cd302-f5bc-4c57-9900-1b6ddb8c5e25" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused"
Oct 30 21:56:12.246946 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:56:12.246364    4473 patch_prober.go:29] interesting pod/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-mg8sv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body=
Oct 30 21:56:12.246946 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:56:12.246430    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-mg8sv" podUID="117cd302-f5bc-4c57-9900-1b6ddb8c5e25" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused"
Oct 30 21:56:13.244576 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:56:13.244529    4473 patch_prober.go:29] interesting pod/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-mg8sv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body=
Oct 30 21:56:13.245145 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:56:13.244615    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-mg8sv" podUID="117cd302-f5bc-4c57-9900-1b6ddb8c5e25" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused"
Oct 30 21:56:14.249385 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:56:14.249345    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-mg8sv"
Oct 30 21:56:14.249844 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:56:14.249655    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-mg8sv"
Oct 30 21:56:14.250364 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:56:14.250348    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-mg8sv"
Oct 30 21:56:20.325348 ip-10-0-140-18 kubenswrapper[4473]: E1030 21:56:20.325312    4473 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fncvg" containerName="extractor"
Oct 30 21:56:50.328129 ip-10-0-140-18 kubenswrapper[4473]: E1030 21:56:50.328052    4473 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fncvg" containerName="extractor"
Oct 30 21:57:08.848580 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:57:08.848535    4473 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w6-29cbad/premium-w6-29cbad-kserve-6c9f8d6899-qtv6c"]
Oct 30 21:57:08.852756 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:57:08.852728    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-29cbad/premium-w6-29cbad-kserve-6c9f8d6899-qtv6c"
Oct 30 21:57:08.924784 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:57:08.924746    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w6-29cbad/premium-w6-29cbad-kserve-6c9f8d6899-qtv6c"]
Oct 30 21:57:08.950530 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:57:08.950492    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/381b11f6-3fde-4de7-9ad3-7c0891e76ba1-home\") pod \"premium-w6-29cbad-kserve-6c9f8d6899-qtv6c\" (UID: \"381b11f6-3fde-4de7-9ad3-7c0891e76ba1\") " pod="e2e-models-e2e-worker-w6-29cbad/premium-w6-29cbad-kserve-6c9f8d6899-qtv6c"
Oct 30 21:57:08.950530 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:57:08.950535    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/381b11f6-3fde-4de7-9ad3-7c0891e76ba1-tmp-dir\") pod \"premium-w6-29cbad-kserve-6c9f8d6899-qtv6c\" (UID: \"381b11f6-3fde-4de7-9ad3-7c0891e76ba1\") " pod="e2e-models-e2e-worker-w6-29cbad/premium-w6-29cbad-kserve-6c9f8d6899-qtv6c"
Oct 30 21:57:08.950789 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:57:08.950665    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d5hwj\" (UniqueName: \"kubernetes.io/projected/381b11f6-3fde-4de7-9ad3-7c0891e76ba1-kube-api-access-d5hwj\") pod \"premium-w6-29cbad-kserve-6c9f8d6899-qtv6c\" (UID: \"381b11f6-3fde-4de7-9ad3-7c0891e76ba1\") " pod="e2e-models-e2e-worker-w6-29cbad/premium-w6-29cbad-kserve-6c9f8d6899-qtv6c"
Oct 30 21:57:08.950789 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:57:08.950704    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/381b11f6-3fde-4de7-9ad3-7c0891e76ba1-dshm\") pod \"premium-w6-29cbad-kserve-6c9f8d6899-qtv6c\" (UID: \"381b11f6-3fde-4de7-9ad3-7c0891e76ba1\") " pod="e2e-models-e2e-worker-w6-29cbad/premium-w6-29cbad-kserve-6c9f8d6899-qtv6c"
Oct 30 21:57:08.950789 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:57:08.950731    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/381b11f6-3fde-4de7-9ad3-7c0891e76ba1-tls-certs\") pod \"premium-w6-29cbad-kserve-6c9f8d6899-qtv6c\" (UID: \"381b11f6-3fde-4de7-9ad3-7c0891e76ba1\") " pod="e2e-models-e2e-worker-w6-29cbad/premium-w6-29cbad-kserve-6c9f8d6899-qtv6c"
Oct 30 21:57:08.950965 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:57:08.950823    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/381b11f6-3fde-4de7-9ad3-7c0891e76ba1-model-cache\") pod \"premium-w6-29cbad-kserve-6c9f8d6899-qtv6c\" (UID: \"381b11f6-3fde-4de7-9ad3-7c0891e76ba1\") " pod="e2e-models-e2e-worker-w6-29cbad/premium-w6-29cbad-kserve-6c9f8d6899-qtv6c"
Oct 30 21:57:08.958610 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:57:08.958582    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w6-29cbad\"/\"default-dockercfg-w89h2\""
Oct 30 21:57:08.958754 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:57:08.958626    4473 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w6-29cbad\"/\"openshift-service-ca.crt\""
Oct 30 21:57:08.958754 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:57:08.958669    4473 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w6-29cbad\"/\"kube-root-ca.crt\""
Oct 30 21:57:08.958833 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:57:08.958755    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w6-29cbad\"/\"premium-w6-29cbad-kserve-self-signed-certs\""
Oct 30 21:57:09.051892 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:57:09.051842    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/381b11f6-3fde-4de7-9ad3-7c0891e76ba1-home\") pod \"premium-w6-29cbad-kserve-6c9f8d6899-qtv6c\" (UID: \"381b11f6-3fde-4de7-9ad3-7c0891e76ba1\") " pod="e2e-models-e2e-worker-w6-29cbad/premium-w6-29cbad-kserve-6c9f8d6899-qtv6c"
Oct 30 21:57:09.051892 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:57:09.051896    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/381b11f6-3fde-4de7-9ad3-7c0891e76ba1-tmp-dir\") pod \"premium-w6-29cbad-kserve-6c9f8d6899-qtv6c\" (UID: \"381b11f6-3fde-4de7-9ad3-7c0891e76ba1\") " pod="e2e-models-e2e-worker-w6-29cbad/premium-w6-29cbad-kserve-6c9f8d6899-qtv6c"
Oct 30 21:57:09.052186 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:57:09.051984    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d5hwj\" (UniqueName: \"kubernetes.io/projected/381b11f6-3fde-4de7-9ad3-7c0891e76ba1-kube-api-access-d5hwj\") pod \"premium-w6-29cbad-kserve-6c9f8d6899-qtv6c\" (UID: \"381b11f6-3fde-4de7-9ad3-7c0891e76ba1\") " pod="e2e-models-e2e-worker-w6-29cbad/premium-w6-29cbad-kserve-6c9f8d6899-qtv6c"
Oct 30 21:57:09.052186 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:57:09.052012    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/381b11f6-3fde-4de7-9ad3-7c0891e76ba1-dshm\") pod \"premium-w6-29cbad-kserve-6c9f8d6899-qtv6c\" (UID: \"381b11f6-3fde-4de7-9ad3-7c0891e76ba1\") " pod="e2e-models-e2e-worker-w6-29cbad/premium-w6-29cbad-kserve-6c9f8d6899-qtv6c"
Oct 30 21:57:09.052186 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:57:09.052106    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/381b11f6-3fde-4de7-9ad3-7c0891e76ba1-tls-certs\") pod \"premium-w6-29cbad-kserve-6c9f8d6899-qtv6c\" (UID: \"381b11f6-3fde-4de7-9ad3-7c0891e76ba1\") " pod="e2e-models-e2e-worker-w6-29cbad/premium-w6-29cbad-kserve-6c9f8d6899-qtv6c"
Oct 30 21:57:09.052186 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:57:09.052179    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/381b11f6-3fde-4de7-9ad3-7c0891e76ba1-model-cache\") pod \"premium-w6-29cbad-kserve-6c9f8d6899-qtv6c\" (UID: \"381b11f6-3fde-4de7-9ad3-7c0891e76ba1\") " pod="e2e-models-e2e-worker-w6-29cbad/premium-w6-29cbad-kserve-6c9f8d6899-qtv6c"
Oct 30 21:57:09.052559 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:57:09.052462    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/381b11f6-3fde-4de7-9ad3-7c0891e76ba1-tmp-dir\") pod \"premium-w6-29cbad-kserve-6c9f8d6899-qtv6c\" (UID: \"381b11f6-3fde-4de7-9ad3-7c0891e76ba1\") " pod="e2e-models-e2e-worker-w6-29cbad/premium-w6-29cbad-kserve-6c9f8d6899-qtv6c"
Oct 30 21:57:09.052559 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:57:09.052518    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/381b11f6-3fde-4de7-9ad3-7c0891e76ba1-model-cache\") pod \"premium-w6-29cbad-kserve-6c9f8d6899-qtv6c\" (UID: \"381b11f6-3fde-4de7-9ad3-7c0891e76ba1\") " pod="e2e-models-e2e-worker-w6-29cbad/premium-w6-29cbad-kserve-6c9f8d6899-qtv6c"
Oct 30 21:57:09.052739 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:57:09.052620    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/381b11f6-3fde-4de7-9ad3-7c0891e76ba1-home\") pod \"premium-w6-29cbad-kserve-6c9f8d6899-qtv6c\" (UID: \"381b11f6-3fde-4de7-9ad3-7c0891e76ba1\") " pod="e2e-models-e2e-worker-w6-29cbad/premium-w6-29cbad-kserve-6c9f8d6899-qtv6c"
Oct 30 21:57:09.054721 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:57:09.054696    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/381b11f6-3fde-4de7-9ad3-7c0891e76ba1-dshm\") pod \"premium-w6-29cbad-kserve-6c9f8d6899-qtv6c\" (UID: \"381b11f6-3fde-4de7-9ad3-7c0891e76ba1\") " pod="e2e-models-e2e-worker-w6-29cbad/premium-w6-29cbad-kserve-6c9f8d6899-qtv6c"
Oct 30 21:57:09.054898 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:57:09.054881    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/381b11f6-3fde-4de7-9ad3-7c0891e76ba1-tls-certs\") pod \"premium-w6-29cbad-kserve-6c9f8d6899-qtv6c\" (UID: \"381b11f6-3fde-4de7-9ad3-7c0891e76ba1\") " pod="e2e-models-e2e-worker-w6-29cbad/premium-w6-29cbad-kserve-6c9f8d6899-qtv6c"
Oct 30 21:57:09.113004 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:57:09.112893    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d5hwj\" (UniqueName: \"kubernetes.io/projected/381b11f6-3fde-4de7-9ad3-7c0891e76ba1-kube-api-access-d5hwj\") pod \"premium-w6-29cbad-kserve-6c9f8d6899-qtv6c\" (UID: \"381b11f6-3fde-4de7-9ad3-7c0891e76ba1\") " pod="e2e-models-e2e-worker-w6-29cbad/premium-w6-29cbad-kserve-6c9f8d6899-qtv6c"
Oct 30 21:57:09.180011 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:57:09.179967    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-29cbad/premium-w6-29cbad-kserve-6c9f8d6899-qtv6c"
Oct 30 21:57:09.568546 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:57:09.568519    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w6-29cbad/premium-w6-29cbad-kserve-6c9f8d6899-qtv6c"]
Oct 30 21:57:09.700624 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:57:09.700586    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-29cbad/premium-w6-29cbad-kserve-6c9f8d6899-qtv6c" event={"ID":"381b11f6-3fde-4de7-9ad3-7c0891e76ba1","Type":"ContainerStarted","Data":"4bfc1fcafc15d27d5fc2e5618f2fc39247300c9d78255d4ee6935452201c747f"}
Oct 30 21:57:09.700783 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:57:09.700636    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-29cbad/premium-w6-29cbad-kserve-6c9f8d6899-qtv6c" event={"ID":"381b11f6-3fde-4de7-9ad3-7c0891e76ba1","Type":"ContainerStarted","Data":"a079d20c320af5c38c2e26ab1736f16a9bc42b6793b728b7d62ff8f25e0104ed"}
Oct 30 21:57:09.829847 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:57:09.829724    4473 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w6-29cbad/premium-w6-29cbad-kserve-6c9f8d6899-qtv6c" podStartSLOduration=1.829710082 podStartE2EDuration="1.829710082s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:57:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:57:09.829178259 +0000 UTC m=+875.602546747" watchObservedRunningTime="2026-10-30 21:57:09.829710082 +0000 UTC m=+875.603078602"
Oct 30 21:57:10.180828 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:57:10.180783    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-29cbad/premium-w6-29cbad-kserve-6c9f8d6899-qtv6c"
Oct 30 21:57:19.187946 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:57:19.187599    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w6-29cbad/premium-w6-29cbad-kserve-6c9f8d6899-qtv6c"
Oct 30 21:57:19.221313 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:57:19.221279    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w6-29cbad/premium-w6-29cbad-kserve-6c9f8d6899-qtv6c"
Oct 30 21:57:19.771091 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:57:19.771060    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-29cbad/premium-w6-29cbad-kserve-6c9f8d6899-qtv6c"
Oct 30 21:57:20.323681 ip-10-0-140-18 kubenswrapper[4473]: E1030 21:57:20.323648    4473 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fncvg" containerName="extractor"
Oct 30 21:57:37.222220 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:57:37.222183    4473 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-jvp7n"]
Oct 30 21:57:37.226236 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:57:37.226208    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-jvp7n"
Oct 30 21:57:37.279603 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:57:37.279568    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w6-29cbad-openshift-default-dockercfg-c5s8g\""
Oct 30 21:57:37.332734 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:57:37.332692    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-jvp7n"]
Oct 30 21:57:37.406907 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:57:37.406870    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/091485c0-17be-49be-a5bc-a26a9b2e89a3-istio-token\") pod \"e2e-worker-w6-29cbad-openshift-default-6d74d46b79-jvp7n\" (UID: \"091485c0-17be-49be-a5bc-a26a9b2e89a3\") " pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-jvp7n"
Oct 30 21:57:37.407118 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:57:37.406957    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7mh8b\" (UniqueName: \"kubernetes.io/projected/091485c0-17be-49be-a5bc-a26a9b2e89a3-kube-api-access-7mh8b\") pod \"e2e-worker-w6-29cbad-openshift-default-6d74d46b79-jvp7n\" (UID: \"091485c0-17be-49be-a5bc-a26a9b2e89a3\") " pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-jvp7n"
Oct 30 21:57:37.407118 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:57:37.406991    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/091485c0-17be-49be-a5bc-a26a9b2e89a3-credential-socket\") pod \"e2e-worker-w6-29cbad-openshift-default-6d74d46b79-jvp7n\" (UID: \"091485c0-17be-49be-a5bc-a26a9b2e89a3\") " pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-jvp7n"
Oct 30 21:57:37.407118 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:57:37.407017    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/091485c0-17be-49be-a5bc-a26a9b2e89a3-istio-envoy\") pod \"e2e-worker-w6-29cbad-openshift-default-6d74d46b79-jvp7n\" (UID: \"091485c0-17be-49be-a5bc-a26a9b2e89a3\") " pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-jvp7n"
Oct 30 21:57:37.407118 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:57:37.407106    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/091485c0-17be-49be-a5bc-a26a9b2e89a3-workload-socket\") pod \"e2e-worker-w6-29cbad-openshift-default-6d74d46b79-jvp7n\" (UID: \"091485c0-17be-49be-a5bc-a26a9b2e89a3\") " pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-jvp7n"
Oct 30 21:57:37.407288 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:57:37.407150    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/091485c0-17be-49be-a5bc-a26a9b2e89a3-istio-podinfo\") pod \"e2e-worker-w6-29cbad-openshift-default-6d74d46b79-jvp7n\" (UID: \"091485c0-17be-49be-a5bc-a26a9b2e89a3\") " pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-jvp7n"
Oct 30 21:57:37.407288 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:57:37.407176    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/091485c0-17be-49be-a5bc-a26a9b2e89a3-workload-certs\") pod \"e2e-worker-w6-29cbad-openshift-default-6d74d46b79-jvp7n\" (UID: \"091485c0-17be-49be-a5bc-a26a9b2e89a3\") " pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-jvp7n"
Oct 30 21:57:37.407288 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:57:37.407273    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/091485c0-17be-49be-a5bc-a26a9b2e89a3-istio-data\") pod \"e2e-worker-w6-29cbad-openshift-default-6d74d46b79-jvp7n\" (UID: \"091485c0-17be-49be-a5bc-a26a9b2e89a3\") " pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-jvp7n"
Oct 30 21:57:37.407399 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:57:37.407290    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/091485c0-17be-49be-a5bc-a26a9b2e89a3-istiod-ca-cert\") pod \"e2e-worker-w6-29cbad-openshift-default-6d74d46b79-jvp7n\" (UID: \"091485c0-17be-49be-a5bc-a26a9b2e89a3\") " pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-jvp7n"
Oct 30 21:57:37.508345 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:57:37.508251    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/091485c0-17be-49be-a5bc-a26a9b2e89a3-istio-token\") pod \"e2e-worker-w6-29cbad-openshift-default-6d74d46b79-jvp7n\" (UID: \"091485c0-17be-49be-a5bc-a26a9b2e89a3\") " pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-jvp7n"
Oct 30 21:57:37.508345 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:57:37.508304    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7mh8b\" (UniqueName: \"kubernetes.io/projected/091485c0-17be-49be-a5bc-a26a9b2e89a3-kube-api-access-7mh8b\") pod \"e2e-worker-w6-29cbad-openshift-default-6d74d46b79-jvp7n\" (UID: \"091485c0-17be-49be-a5bc-a26a9b2e89a3\") " pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-jvp7n"
Oct 30 21:57:37.508551 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:57:37.508424    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/091485c0-17be-49be-a5bc-a26a9b2e89a3-credential-socket\") pod \"e2e-worker-w6-29cbad-openshift-default-6d74d46b79-jvp7n\" (UID: \"091485c0-17be-49be-a5bc-a26a9b2e89a3\") " pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-jvp7n"
Oct 30 21:57:37.508551 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:57:37.508487    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/091485c0-17be-49be-a5bc-a26a9b2e89a3-istio-envoy\") pod \"e2e-worker-w6-29cbad-openshift-default-6d74d46b79-jvp7n\" (UID: \"091485c0-17be-49be-a5bc-a26a9b2e89a3\") " pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-jvp7n"
Oct 30 21:57:37.508551 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:57:37.508514    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/091485c0-17be-49be-a5bc-a26a9b2e89a3-workload-socket\") pod \"e2e-worker-w6-29cbad-openshift-default-6d74d46b79-jvp7n\" (UID: \"091485c0-17be-49be-a5bc-a26a9b2e89a3\") " pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-jvp7n"
Oct 30 21:57:37.508689 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:57:37.508565    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/091485c0-17be-49be-a5bc-a26a9b2e89a3-istio-podinfo\") pod \"e2e-worker-w6-29cbad-openshift-default-6d74d46b79-jvp7n\" (UID: \"091485c0-17be-49be-a5bc-a26a9b2e89a3\") " pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-jvp7n"
Oct 30 21:57:37.508689 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:57:37.508636    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/091485c0-17be-49be-a5bc-a26a9b2e89a3-workload-certs\") pod \"e2e-worker-w6-29cbad-openshift-default-6d74d46b79-jvp7n\" (UID: \"091485c0-17be-49be-a5bc-a26a9b2e89a3\") " pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-jvp7n"
Oct 30 21:57:37.508794 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:57:37.508728    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/091485c0-17be-49be-a5bc-a26a9b2e89a3-istio-data\") pod \"e2e-worker-w6-29cbad-openshift-default-6d74d46b79-jvp7n\" (UID: \"091485c0-17be-49be-a5bc-a26a9b2e89a3\") " pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-jvp7n"
Oct 30 21:57:37.508794 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:57:37.508754    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/091485c0-17be-49be-a5bc-a26a9b2e89a3-istiod-ca-cert\") pod \"e2e-worker-w6-29cbad-openshift-default-6d74d46b79-jvp7n\" (UID: \"091485c0-17be-49be-a5bc-a26a9b2e89a3\") " pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-jvp7n"
Oct 30 21:57:37.508903 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:57:37.508877    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/091485c0-17be-49be-a5bc-a26a9b2e89a3-credential-socket\") pod \"e2e-worker-w6-29cbad-openshift-default-6d74d46b79-jvp7n\" (UID: \"091485c0-17be-49be-a5bc-a26a9b2e89a3\") " pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-jvp7n"
Oct 30 21:57:37.509019 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:57:37.508971    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/091485c0-17be-49be-a5bc-a26a9b2e89a3-workload-socket\") pod \"e2e-worker-w6-29cbad-openshift-default-6d74d46b79-jvp7n\" (UID: \"091485c0-17be-49be-a5bc-a26a9b2e89a3\") " pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-jvp7n"
Oct 30 21:57:37.509084 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:57:37.509040    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/091485c0-17be-49be-a5bc-a26a9b2e89a3-workload-certs\") pod \"e2e-worker-w6-29cbad-openshift-default-6d74d46b79-jvp7n\" (UID: \"091485c0-17be-49be-a5bc-a26a9b2e89a3\") " pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-jvp7n"
Oct 30 21:57:37.509394 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:57:37.509300    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/091485c0-17be-49be-a5bc-a26a9b2e89a3-istio-data\") pod \"e2e-worker-w6-29cbad-openshift-default-6d74d46b79-jvp7n\" (UID: \"091485c0-17be-49be-a5bc-a26a9b2e89a3\") " pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-jvp7n"
Oct 30 21:57:37.509541 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:57:37.509519    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/091485c0-17be-49be-a5bc-a26a9b2e89a3-istiod-ca-cert\") pod \"e2e-worker-w6-29cbad-openshift-default-6d74d46b79-jvp7n\" (UID: \"091485c0-17be-49be-a5bc-a26a9b2e89a3\") " pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-jvp7n"
Oct 30 21:57:37.510984 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:57:37.510958    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/091485c0-17be-49be-a5bc-a26a9b2e89a3-istio-envoy\") pod \"e2e-worker-w6-29cbad-openshift-default-6d74d46b79-jvp7n\" (UID: \"091485c0-17be-49be-a5bc-a26a9b2e89a3\") " pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-jvp7n"
Oct 30 21:57:37.511202 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:57:37.511187    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/091485c0-17be-49be-a5bc-a26a9b2e89a3-istio-podinfo\") pod \"e2e-worker-w6-29cbad-openshift-default-6d74d46b79-jvp7n\" (UID: \"091485c0-17be-49be-a5bc-a26a9b2e89a3\") " pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-jvp7n"
Oct 30 21:57:37.566535 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:57:37.566493    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/091485c0-17be-49be-a5bc-a26a9b2e89a3-istio-token\") pod \"e2e-worker-w6-29cbad-openshift-default-6d74d46b79-jvp7n\" (UID: \"091485c0-17be-49be-a5bc-a26a9b2e89a3\") " pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-jvp7n"
Oct 30 21:57:37.566717 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:57:37.566561    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7mh8b\" (UniqueName: \"kubernetes.io/projected/091485c0-17be-49be-a5bc-a26a9b2e89a3-kube-api-access-7mh8b\") pod \"e2e-worker-w6-29cbad-openshift-default-6d74d46b79-jvp7n\" (UID: \"091485c0-17be-49be-a5bc-a26a9b2e89a3\") " pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-jvp7n"
Oct 30 21:57:37.840271 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:57:37.840183    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-jvp7n"
Oct 30 21:57:38.274880 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:57:38.274841    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-jvp7n"]
Oct 30 21:57:38.311451 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:57:38.311419    4473 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod091485c0_17be_49be_a5bc_a26a9b2e89a3.slice/crio-9fc7bdbe9a16b5ddc1a9d226593c8e175b3960949b6c5d985047fc88bbe8fac5 WatchSource:0}: Error finding container 9fc7bdbe9a16b5ddc1a9d226593c8e175b3960949b6c5d985047fc88bbe8fac5: Status 404 returned error can't find the container with id 9fc7bdbe9a16b5ddc1a9d226593c8e175b3960949b6c5d985047fc88bbe8fac5
Oct 30 21:57:38.313735 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:57:38.313696    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:57:38.313825 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:57:38.313773    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:57:38.313825 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:57:38.313808    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:57:38.886554 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:57:38.886519    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-jvp7n" event={"ID":"091485c0-17be-49be-a5bc-a26a9b2e89a3","Type":"ContainerStarted","Data":"4992977dde23e771b8de76f1efc81470ebec5cc87566438728b5abe889ea2278"}
Oct 30 21:57:38.886768 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:57:38.886599    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-jvp7n" event={"ID":"091485c0-17be-49be-a5bc-a26a9b2e89a3","Type":"ContainerStarted","Data":"9fc7bdbe9a16b5ddc1a9d226593c8e175b3960949b6c5d985047fc88bbe8fac5"}
Oct 30 21:57:39.062519 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:57:39.062460    4473 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-jvp7n" podStartSLOduration=2.062446247 podStartE2EDuration="2.062446247s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:57:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:57:39.060776416 +0000 UTC m=+904.834144904" watchObservedRunningTime="2026-10-30 21:57:39.062446247 +0000 UTC m=+904.835814733"
Oct 30 21:57:39.847646 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:57:39.846363    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-jvp7n"
Oct 30 21:57:39.851177 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:57:39.851144    4473 patch_prober.go:29] interesting pod/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-jvp7n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body=
Oct 30 21:57:39.851442 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:57:39.851407    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-jvp7n" podUID="091485c0-17be-49be-a5bc-a26a9b2e89a3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused"
Oct 30 21:57:40.842947 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:57:40.841843    4473 patch_prober.go:29] interesting pod/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-jvp7n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body=
Oct 30 21:57:40.842947 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:57:40.841948    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-jvp7n" podUID="091485c0-17be-49be-a5bc-a26a9b2e89a3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused"
Oct 30 21:57:41.845949 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:57:41.845054    4473 patch_prober.go:29] interesting pod/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-jvp7n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body=
Oct 30 21:57:41.845949 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:57:41.845138    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-jvp7n" podUID="091485c0-17be-49be-a5bc-a26a9b2e89a3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused"
Oct 30 21:57:42.843947 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:57:42.843340    4473 patch_prober.go:29] interesting pod/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-jvp7n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body=
Oct 30 21:57:42.843947 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:57:42.843421    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-jvp7n" podUID="091485c0-17be-49be-a5bc-a26a9b2e89a3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused"
Oct 30 21:57:43.842402 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:57:43.842359    4473 patch_prober.go:29] interesting pod/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-jvp7n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body=
Oct 30 21:57:43.843102 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:57:43.843063    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-jvp7n" podUID="091485c0-17be-49be-a5bc-a26a9b2e89a3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused"
Oct 30 21:57:44.850954 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:57:44.849596    4473 patch_prober.go:29] interesting pod/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-jvp7n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body=
Oct 30 21:57:44.850954 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:57:44.849659    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-jvp7n" podUID="091485c0-17be-49be-a5bc-a26a9b2e89a3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused"
Oct 30 21:57:45.845672 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:57:45.845643    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-jvp7n"
Oct 30 21:57:45.846075 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:57:45.846047    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-jvp7n"
Oct 30 21:57:45.846900 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:57:45.846885    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-jvp7n"
Oct 30 21:57:50.324565 ip-10-0-140-18 kubenswrapper[4473]: E1030 21:57:50.324526    4473 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fncvg" containerName="extractor"
Oct 30 21:57:53.417599 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:57:53.417565    4473 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-fnz9z"]
Oct 30 21:57:53.421567 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:57:53.421547    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-fnz9z"
Oct 30 21:57:53.475029 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:57:53.474994    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w0-dde1e8-openshift-default-dockercfg-5l6vx\""
Oct 30 21:57:53.507139 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:57:53.507098    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f46a30dc-0abe-4906-8b58-c439a0bff711-credential-socket\") pod \"e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-fnz9z\" (UID: \"f46a30dc-0abe-4906-8b58-c439a0bff711\") " pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-fnz9z"
Oct 30 21:57:53.507343 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:57:53.507150    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f46a30dc-0abe-4906-8b58-c439a0bff711-workload-certs\") pod \"e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-fnz9z\" (UID: \"f46a30dc-0abe-4906-8b58-c439a0bff711\") " pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-fnz9z"
Oct 30 21:57:53.507343 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:57:53.507292    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4ktvc\" (UniqueName: \"kubernetes.io/projected/f46a30dc-0abe-4906-8b58-c439a0bff711-kube-api-access-4ktvc\") pod \"e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-fnz9z\" (UID: \"f46a30dc-0abe-4906-8b58-c439a0bff711\") " pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-fnz9z"
Oct 30 21:57:53.507343 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:57:53.507334    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f46a30dc-0abe-4906-8b58-c439a0bff711-istio-token\") pod \"e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-fnz9z\" (UID: \"f46a30dc-0abe-4906-8b58-c439a0bff711\") " pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-fnz9z"
Oct 30 21:57:53.507510 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:57:53.507433    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f46a30dc-0abe-4906-8b58-c439a0bff711-istio-envoy\") pod \"e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-fnz9z\" (UID: \"f46a30dc-0abe-4906-8b58-c439a0bff711\") " pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-fnz9z"
Oct 30 21:57:53.507510 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:57:53.507486    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f46a30dc-0abe-4906-8b58-c439a0bff711-istio-data\") pod \"e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-fnz9z\" (UID: \"f46a30dc-0abe-4906-8b58-c439a0bff711\") " pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-fnz9z"
Oct 30 21:57:53.507601 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:57:53.507584    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f46a30dc-0abe-4906-8b58-c439a0bff711-istio-podinfo\") pod \"e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-fnz9z\" (UID: \"f46a30dc-0abe-4906-8b58-c439a0bff711\") " pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-fnz9z"
Oct 30 21:57:53.507637 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:57:53.507612    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f46a30dc-0abe-4906-8b58-c439a0bff711-istiod-ca-cert\") pod \"e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-fnz9z\" (UID: \"f46a30dc-0abe-4906-8b58-c439a0bff711\") " pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-fnz9z"
Oct 30 21:57:53.507678 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:57:53.507636    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f46a30dc-0abe-4906-8b58-c439a0bff711-workload-socket\") pod \"e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-fnz9z\" (UID: \"f46a30dc-0abe-4906-8b58-c439a0bff711\") " pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-fnz9z"
Oct 30 21:57:53.528412 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:57:53.528375    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-fnz9z"]
Oct 30 21:57:53.608537 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:57:53.608495    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f46a30dc-0abe-4906-8b58-c439a0bff711-istio-envoy\") pod \"e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-fnz9z\" (UID: \"f46a30dc-0abe-4906-8b58-c439a0bff711\") " pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-fnz9z"
Oct 30 21:57:53.608537 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:57:53.608544    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f46a30dc-0abe-4906-8b58-c439a0bff711-istio-data\") pod \"e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-fnz9z\" (UID: \"f46a30dc-0abe-4906-8b58-c439a0bff711\") " pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-fnz9z"
Oct 30 21:57:53.608799 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:57:53.608612    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f46a30dc-0abe-4906-8b58-c439a0bff711-istio-podinfo\") pod \"e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-fnz9z\" (UID: \"f46a30dc-0abe-4906-8b58-c439a0bff711\") " pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-fnz9z"
Oct 30 21:57:53.608799 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:57:53.608640    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f46a30dc-0abe-4906-8b58-c439a0bff711-istiod-ca-cert\") pod \"e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-fnz9z\" (UID: \"f46a30dc-0abe-4906-8b58-c439a0bff711\") " pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-fnz9z"
Oct 30 21:57:53.608799 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:57:53.608672    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f46a30dc-0abe-4906-8b58-c439a0bff711-workload-socket\") pod \"e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-fnz9z\" (UID: \"f46a30dc-0abe-4906-8b58-c439a0bff711\") " pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-fnz9z"
Oct 30 21:57:53.608799 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:57:53.608702    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f46a30dc-0abe-4906-8b58-c439a0bff711-credential-socket\") pod \"e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-fnz9z\" (UID: \"f46a30dc-0abe-4906-8b58-c439a0bff711\") " pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-fnz9z"
Oct 30 21:57:53.608799 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:57:53.608741    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f46a30dc-0abe-4906-8b58-c439a0bff711-workload-certs\") pod \"e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-fnz9z\" (UID: \"f46a30dc-0abe-4906-8b58-c439a0bff711\") " pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-fnz9z"
Oct 30 21:57:53.609097 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:57:53.608808    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4ktvc\" (UniqueName: \"kubernetes.io/projected/f46a30dc-0abe-4906-8b58-c439a0bff711-kube-api-access-4ktvc\") pod \"e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-fnz9z\" (UID: \"f46a30dc-0abe-4906-8b58-c439a0bff711\") " pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-fnz9z"
Oct 30 21:57:53.609097 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:57:53.608857    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f46a30dc-0abe-4906-8b58-c439a0bff711-istio-token\") pod \"e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-fnz9z\" (UID: \"f46a30dc-0abe-4906-8b58-c439a0bff711\") " pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-fnz9z"
Oct 30 21:57:53.609207 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:57:53.609097    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f46a30dc-0abe-4906-8b58-c439a0bff711-istio-data\") pod \"e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-fnz9z\" (UID: \"f46a30dc-0abe-4906-8b58-c439a0bff711\") " pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-fnz9z"
Oct 30 21:57:53.609207 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:57:53.609178    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f46a30dc-0abe-4906-8b58-c439a0bff711-workload-socket\") pod \"e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-fnz9z\" (UID: \"f46a30dc-0abe-4906-8b58-c439a0bff711\") " pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-fnz9z"
Oct 30 21:57:53.609484 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:57:53.609462    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f46a30dc-0abe-4906-8b58-c439a0bff711-istiod-ca-cert\") pod \"e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-fnz9z\" (UID: \"f46a30dc-0abe-4906-8b58-c439a0bff711\") " pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-fnz9z"
Oct 30 21:57:53.609563 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:57:53.609419    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f46a30dc-0abe-4906-8b58-c439a0bff711-workload-certs\") pod \"e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-fnz9z\" (UID: \"f46a30dc-0abe-4906-8b58-c439a0bff711\") " pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-fnz9z"
Oct 30 21:57:53.609719 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:57:53.609697    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f46a30dc-0abe-4906-8b58-c439a0bff711-credential-socket\") pod \"e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-fnz9z\" (UID: \"f46a30dc-0abe-4906-8b58-c439a0bff711\") " pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-fnz9z"
Oct 30 21:57:53.611090 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:57:53.611062    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f46a30dc-0abe-4906-8b58-c439a0bff711-istio-envoy\") pod \"e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-fnz9z\" (UID: \"f46a30dc-0abe-4906-8b58-c439a0bff711\") " pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-fnz9z"
Oct 30 21:57:53.611458 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:57:53.611434    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f46a30dc-0abe-4906-8b58-c439a0bff711-istio-podinfo\") pod \"e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-fnz9z\" (UID: \"f46a30dc-0abe-4906-8b58-c439a0bff711\") " pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-fnz9z"
Oct 30 21:57:53.666318 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:57:53.666286    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f46a30dc-0abe-4906-8b58-c439a0bff711-istio-token\") pod \"e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-fnz9z\" (UID: \"f46a30dc-0abe-4906-8b58-c439a0bff711\") " pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-fnz9z"
Oct 30 21:57:53.666471 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:57:53.666450    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4ktvc\" (UniqueName: \"kubernetes.io/projected/f46a30dc-0abe-4906-8b58-c439a0bff711-kube-api-access-4ktvc\") pod \"e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-fnz9z\" (UID: \"f46a30dc-0abe-4906-8b58-c439a0bff711\") " pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-fnz9z"
Oct 30 21:57:53.736662 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:57:53.736567    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-fnz9z"
Oct 30 21:57:54.167955 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:57:54.167885    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-fnz9z"]
Oct 30 21:57:54.207046 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:57:54.207015    4473 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf46a30dc_0abe_4906_8b58_c439a0bff711.slice/crio-afce6ad9496157a686c547656e42403757a3f7b8094a92fb5132aca01377af1a WatchSource:0}: Error finding container afce6ad9496157a686c547656e42403757a3f7b8094a92fb5132aca01377af1a: Status 404 returned error can't find the container with id afce6ad9496157a686c547656e42403757a3f7b8094a92fb5132aca01377af1a
Oct 30 21:57:54.209523 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:57:54.209488    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:57:54.209599 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:57:54.209568    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:57:54.209638 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:57:54.209619    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:57:54.979804 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:57:54.979762    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-fnz9z" event={"ID":"f46a30dc-0abe-4906-8b58-c439a0bff711","Type":"ContainerStarted","Data":"deae3cac1ce1326b33a78cf69d9fe0fb567fad5a2944340b2c89679e7406e439"}
Oct 30 21:57:54.980274 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:57:54.979841    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-fnz9z" event={"ID":"f46a30dc-0abe-4906-8b58-c439a0bff711","Type":"ContainerStarted","Data":"afce6ad9496157a686c547656e42403757a3f7b8094a92fb5132aca01377af1a"}
Oct 30 21:57:55.170307 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:57:55.170251    4473 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-fnz9z" podStartSLOduration=2.170236873 podStartE2EDuration="2.170236873s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:57:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:57:55.167510612 +0000 UTC m=+920.940879101" watchObservedRunningTime="2026-10-30 21:57:55.170236873 +0000 UTC m=+920.943605359"
Oct 30 21:57:55.740356 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:57:55.740322    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-fnz9z"
Oct 30 21:57:55.750192 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:57:55.750135    4473 patch_prober.go:29] interesting pod/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-fnz9z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body=
Oct 30 21:57:55.750376 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:57:55.750239    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-fnz9z" podUID="f46a30dc-0abe-4906-8b58-c439a0bff711" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused"
Oct 30 21:57:56.740491 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:57:56.740452    4473 patch_prober.go:29] interesting pod/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-fnz9z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body=
Oct 30 21:57:56.741168 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:57:56.741134    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-fnz9z" podUID="f46a30dc-0abe-4906-8b58-c439a0bff711" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused"
Oct 30 21:57:57.744955 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:57:57.743336    4473 patch_prober.go:29] interesting pod/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-fnz9z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body=
Oct 30 21:57:57.744955 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:57:57.743411    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-fnz9z" podUID="f46a30dc-0abe-4906-8b58-c439a0bff711" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused"
Oct 30 21:57:58.745948 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:57:58.741410    4473 patch_prober.go:29] interesting pod/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-fnz9z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body=
Oct 30 21:57:58.745948 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:57:58.741472    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-fnz9z" podUID="f46a30dc-0abe-4906-8b58-c439a0bff711" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused"
Oct 30 21:57:59.748948 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:57:59.748276    4473 patch_prober.go:29] interesting pod/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-fnz9z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body=
Oct 30 21:57:59.748948 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:57:59.748343    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-fnz9z" podUID="f46a30dc-0abe-4906-8b58-c439a0bff711" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused"
Oct 30 21:58:00.737933 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:58:00.737882    4473 patch_prober.go:29] interesting pod/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-fnz9z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body=
Oct 30 21:58:00.738101 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:58:00.737982    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-fnz9z" podUID="f46a30dc-0abe-4906-8b58-c439a0bff711" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused"
Oct 30 21:58:01.741878 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:58:01.741849    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-fnz9z"
Oct 30 21:58:01.742416 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:58:01.742185    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-fnz9z"
Oct 30 21:58:01.742997 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:58:01.742980    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-fnz9z"
Oct 30 21:58:02.826100 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:58:02.826065    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-t4dzc"]
Oct 30 21:58:02.826553 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:58:02.826332    4473 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-t4dzc" podUID="3172740f-9545-4759-ba06-48beeb526631" containerName="istio-proxy" containerID="cri-o://9c90e8240ca667e36a237ce56d20713cfa33931cef6aabbde9155bd226c30fcd" gracePeriod=30
Oct 30 21:58:08.059218 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:58:08.059180    4473 generic.go:356] "Generic (PLEG): container finished" podID="3172740f-9545-4759-ba06-48beeb526631" containerID="9c90e8240ca667e36a237ce56d20713cfa33931cef6aabbde9155bd226c30fcd" exitCode=0
Oct 30 21:58:08.059614 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:58:08.059254    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-t4dzc" event={"ID":"3172740f-9545-4759-ba06-48beeb526631","Type":"ContainerDied","Data":"9c90e8240ca667e36a237ce56d20713cfa33931cef6aabbde9155bd226c30fcd"}
Oct 30 21:58:08.098224 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:58:08.098148    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-t4dzc"
Oct 30 21:58:08.204735 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:58:08.204701    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3172740f-9545-4759-ba06-48beeb526631-kube-api-access-c2l8c\" (UniqueName: \"kubernetes.io/projected/3172740f-9545-4759-ba06-48beeb526631-kube-api-access-c2l8c\") pod \"3172740f-9545-4759-ba06-48beeb526631\" (UID: \"3172740f-9545-4759-ba06-48beeb526631\") "
Oct 30 21:58:08.204987 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:58:08.204770    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3172740f-9545-4759-ba06-48beeb526631-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3172740f-9545-4759-ba06-48beeb526631-workload-certs\") pod \"3172740f-9545-4759-ba06-48beeb526631\" (UID: \"3172740f-9545-4759-ba06-48beeb526631\") "
Oct 30 21:58:08.204987 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:58:08.204800    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3172740f-9545-4759-ba06-48beeb526631-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3172740f-9545-4759-ba06-48beeb526631-istio-data\") pod \"3172740f-9545-4759-ba06-48beeb526631\" (UID: \"3172740f-9545-4759-ba06-48beeb526631\") "
Oct 30 21:58:08.204987 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:58:08.204848    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3172740f-9545-4759-ba06-48beeb526631-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3172740f-9545-4759-ba06-48beeb526631-credential-socket\") pod \"3172740f-9545-4759-ba06-48beeb526631\" (UID: \"3172740f-9545-4759-ba06-48beeb526631\") "
Oct 30 21:58:08.204987 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:58:08.204883    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3172740f-9545-4759-ba06-48beeb526631-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3172740f-9545-4759-ba06-48beeb526631-istio-envoy\") pod \"3172740f-9545-4759-ba06-48beeb526631\" (UID: \"3172740f-9545-4759-ba06-48beeb526631\") "
Oct 30 21:58:08.204987 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:58:08.204912    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/3172740f-9545-4759-ba06-48beeb526631-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3172740f-9545-4759-ba06-48beeb526631-istio-podinfo\") pod \"3172740f-9545-4759-ba06-48beeb526631\" (UID: \"3172740f-9545-4759-ba06-48beeb526631\") "
Oct 30 21:58:08.204987 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:58:08.204961    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3172740f-9545-4759-ba06-48beeb526631-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3172740f-9545-4759-ba06-48beeb526631-workload-socket\") pod \"3172740f-9545-4759-ba06-48beeb526631\" (UID: \"3172740f-9545-4759-ba06-48beeb526631\") "
Oct 30 21:58:08.205319 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:58:08.205095    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3172740f-9545-4759-ba06-48beeb526631-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3172740f-9545-4759-ba06-48beeb526631-istiod-ca-cert\") pod \"3172740f-9545-4759-ba06-48beeb526631\" (UID: \"3172740f-9545-4759-ba06-48beeb526631\") "
Oct 30 21:58:08.205319 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:58:08.205178    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3172740f-9545-4759-ba06-48beeb526631-istio-token\" (UniqueName: \"kubernetes.io/projected/3172740f-9545-4759-ba06-48beeb526631-istio-token\") pod \"3172740f-9545-4759-ba06-48beeb526631\" (UID: \"3172740f-9545-4759-ba06-48beeb526631\") "
Oct 30 21:58:08.205319 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:58:08.205201    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3172740f-9545-4759-ba06-48beeb526631-workload-certs" pod "3172740f-9545-4759-ba06-48beeb526631" (UID: "3172740f-9545-4759-ba06-48beeb526631"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:58:08.205319 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:58:08.205245    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3172740f-9545-4759-ba06-48beeb526631-credential-socket" pod "3172740f-9545-4759-ba06-48beeb526631" (UID: "3172740f-9545-4759-ba06-48beeb526631"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:58:08.205319 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:58:08.205261    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3172740f-9545-4759-ba06-48beeb526631-istio-data" pod "3172740f-9545-4759-ba06-48beeb526631" (UID: "3172740f-9545-4759-ba06-48beeb526631"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:58:08.205580 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:58:08.205374    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3172740f-9545-4759-ba06-48beeb526631-workload-socket" pod "3172740f-9545-4759-ba06-48beeb526631" (UID: "3172740f-9545-4759-ba06-48beeb526631"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:58:08.205580 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:58:08.205525    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3172740f-9545-4759-ba06-48beeb526631-istiod-ca-cert" pod "3172740f-9545-4759-ba06-48beeb526631" (UID: "3172740f-9545-4759-ba06-48beeb526631"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:58:08.205682 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:58:08.205651    4473 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3172740f-9545-4759-ba06-48beeb526631-workload-certs\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:58:08.205682 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:58:08.205671    4473 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3172740f-9545-4759-ba06-48beeb526631-istio-data\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:58:08.205773 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:58:08.205686    4473 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3172740f-9545-4759-ba06-48beeb526631-credential-socket\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:58:08.205773 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:58:08.205701    4473 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3172740f-9545-4759-ba06-48beeb526631-workload-socket\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:58:08.205773 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:58:08.205715    4473 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3172740f-9545-4759-ba06-48beeb526631-istiod-ca-cert\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:58:08.207350 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:58:08.207321    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3172740f-9545-4759-ba06-48beeb526631-istio-envoy" pod "3172740f-9545-4759-ba06-48beeb526631" (UID: "3172740f-9545-4759-ba06-48beeb526631"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:58:08.207499 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:58:08.207481    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3172740f-9545-4759-ba06-48beeb526631-istio-podinfo" pod "3172740f-9545-4759-ba06-48beeb526631" (UID: "3172740f-9545-4759-ba06-48beeb526631"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:58:08.207548 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:58:08.207492    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3172740f-9545-4759-ba06-48beeb526631-kube-api-access-c2l8c" pod "3172740f-9545-4759-ba06-48beeb526631" (UID: "3172740f-9545-4759-ba06-48beeb526631"). InnerVolumeSpecName "kube-api-access-c2l8c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:58:08.207742 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:58:08.207728    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3172740f-9545-4759-ba06-48beeb526631-istio-token" pod "3172740f-9545-4759-ba06-48beeb526631" (UID: "3172740f-9545-4759-ba06-48beeb526631"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:58:08.307403 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:58:08.307362    4473 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3172740f-9545-4759-ba06-48beeb526631-istio-token\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:58:08.307403 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:58:08.307408    4473 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c2l8c\" (UniqueName: \"kubernetes.io/projected/3172740f-9545-4759-ba06-48beeb526631-kube-api-access-c2l8c\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:58:08.307626 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:58:08.307426    4473 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3172740f-9545-4759-ba06-48beeb526631-istio-envoy\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:58:08.307626 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:58:08.307441    4473 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3172740f-9545-4759-ba06-48beeb526631-istio-podinfo\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:58:09.066413 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:58:09.066383    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-t4dzc"
Oct 30 21:58:09.066830 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:58:09.066412    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-t4dzc" event={"ID":"3172740f-9545-4759-ba06-48beeb526631","Type":"ContainerDied","Data":"ba0c5c6ce3f8cc2a44301f0b36b0f54cd4fc19e64d36f91194efd7efcc6fab74"}
Oct 30 21:58:09.066830 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:58:09.066473    4473 scope.go:122] "RemoveContainer" containerID="9c90e8240ca667e36a237ce56d20713cfa33931cef6aabbde9155bd226c30fcd"
Oct 30 21:58:09.284265 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:58:09.284227    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-t4dzc"]
Oct 30 21:58:09.290284 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:58:09.290253    4473 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-t4dzc"]
Oct 30 21:58:10.357599 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:58:10.357562    4473 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3172740f-9545-4759-ba06-48beeb526631" path="/var/lib/kubelet/pods/3172740f-9545-4759-ba06-48beeb526631/volumes"
Oct 30 21:58:20.325264 ip-10-0-140-18 kubenswrapper[4473]: E1030 21:58:20.325177    4473 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fncvg" containerName="extractor"
Oct 30 21:58:50.324424 ip-10-0-140-18 kubenswrapper[4473]: E1030 21:58:50.324386    4473 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fncvg" containerName="extractor"
Oct 30 21:59:02.860736 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:02.860700    4473 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-jzz7s"]
Oct 30 21:59:02.861598 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:02.861577    4473 cpu_manager.go:414] "containerMap: removing container" podUID="3172740f-9545-4759-ba06-48beeb526631" containerName="istio-proxy"
Oct 30 21:59:02.861710 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:02.861599    4473 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3172740f-9545-4759-ba06-48beeb526631" containerName="istio-proxy"
Oct 30 21:59:02.861765 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:02.861714    4473 memory_manager.go:365] "RemoveStaleState removing state" podUID="3172740f-9545-4759-ba06-48beeb526631" containerName="istio-proxy"
Oct 30 21:59:02.865138 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:02.865118    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-jzz7s"
Oct 30 21:59:02.968449 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:02.968336    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-jzz7s"]
Oct 30 21:59:02.987468 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:02.987432    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c9455dd0-779f-4bcb-aa4f-adb7bd780159-workload-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-jzz7s\" (UID: \"c9455dd0-779f-4bcb-aa4f-adb7bd780159\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-jzz7s"
Oct 30 21:59:02.987677 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:02.987480    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c9455dd0-779f-4bcb-aa4f-adb7bd780159-istio-data\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-jzz7s\" (UID: \"c9455dd0-779f-4bcb-aa4f-adb7bd780159\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-jzz7s"
Oct 30 21:59:02.987753 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:02.987718    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c9455dd0-779f-4bcb-aa4f-adb7bd780159-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-jzz7s\" (UID: \"c9455dd0-779f-4bcb-aa4f-adb7bd780159\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-jzz7s"
Oct 30 21:59:02.987805 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:02.987767    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c9455dd0-779f-4bcb-aa4f-adb7bd780159-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-jzz7s\" (UID: \"c9455dd0-779f-4bcb-aa4f-adb7bd780159\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-jzz7s"
Oct 30 21:59:02.987908 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:02.987881    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r5pz8\" (UniqueName: \"kubernetes.io/projected/c9455dd0-779f-4bcb-aa4f-adb7bd780159-kube-api-access-r5pz8\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-jzz7s\" (UID: \"c9455dd0-779f-4bcb-aa4f-adb7bd780159\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-jzz7s"
Oct 30 21:59:02.988024 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:02.988002    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c9455dd0-779f-4bcb-aa4f-adb7bd780159-workload-certs\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-jzz7s\" (UID: \"c9455dd0-779f-4bcb-aa4f-adb7bd780159\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-jzz7s"
Oct 30 21:59:02.988146 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:02.988125    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c9455dd0-779f-4bcb-aa4f-adb7bd780159-istio-token\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-jzz7s\" (UID: \"c9455dd0-779f-4bcb-aa4f-adb7bd780159\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-jzz7s"
Oct 30 21:59:02.988201 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:02.988171    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c9455dd0-779f-4bcb-aa4f-adb7bd780159-istio-envoy\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-jzz7s\" (UID: \"c9455dd0-779f-4bcb-aa4f-adb7bd780159\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-jzz7s"
Oct 30 21:59:02.988299 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:02.988267    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c9455dd0-779f-4bcb-aa4f-adb7bd780159-credential-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-jzz7s\" (UID: \"c9455dd0-779f-4bcb-aa4f-adb7bd780159\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-jzz7s"
Oct 30 21:59:03.089908 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:03.089866    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c9455dd0-779f-4bcb-aa4f-adb7bd780159-istio-envoy\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-jzz7s\" (UID: \"c9455dd0-779f-4bcb-aa4f-adb7bd780159\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-jzz7s"
Oct 30 21:59:03.090124 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:03.089959    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c9455dd0-779f-4bcb-aa4f-adb7bd780159-credential-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-jzz7s\" (UID: \"c9455dd0-779f-4bcb-aa4f-adb7bd780159\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-jzz7s"
Oct 30 21:59:03.090124 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:03.090017    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c9455dd0-779f-4bcb-aa4f-adb7bd780159-workload-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-jzz7s\" (UID: \"c9455dd0-779f-4bcb-aa4f-adb7bd780159\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-jzz7s"
Oct 30 21:59:03.090124 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:03.090061    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c9455dd0-779f-4bcb-aa4f-adb7bd780159-istio-data\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-jzz7s\" (UID: \"c9455dd0-779f-4bcb-aa4f-adb7bd780159\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-jzz7s"
Oct 30 21:59:03.090294 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:03.090174    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c9455dd0-779f-4bcb-aa4f-adb7bd780159-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-jzz7s\" (UID: \"c9455dd0-779f-4bcb-aa4f-adb7bd780159\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-jzz7s"
Oct 30 21:59:03.090294 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:03.090214    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c9455dd0-779f-4bcb-aa4f-adb7bd780159-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-jzz7s\" (UID: \"c9455dd0-779f-4bcb-aa4f-adb7bd780159\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-jzz7s"
Oct 30 21:59:03.090294 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:03.090248    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r5pz8\" (UniqueName: \"kubernetes.io/projected/c9455dd0-779f-4bcb-aa4f-adb7bd780159-kube-api-access-r5pz8\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-jzz7s\" (UID: \"c9455dd0-779f-4bcb-aa4f-adb7bd780159\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-jzz7s"
Oct 30 21:59:03.090460 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:03.090294    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c9455dd0-779f-4bcb-aa4f-adb7bd780159-workload-certs\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-jzz7s\" (UID: \"c9455dd0-779f-4bcb-aa4f-adb7bd780159\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-jzz7s"
Oct 30 21:59:03.090460 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:03.090379    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c9455dd0-779f-4bcb-aa4f-adb7bd780159-istio-token\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-jzz7s\" (UID: \"c9455dd0-779f-4bcb-aa4f-adb7bd780159\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-jzz7s"
Oct 30 21:59:03.090460 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:03.090444    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c9455dd0-779f-4bcb-aa4f-adb7bd780159-credential-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-jzz7s\" (UID: \"c9455dd0-779f-4bcb-aa4f-adb7bd780159\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-jzz7s"
Oct 30 21:59:03.090990 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:03.090943    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c9455dd0-779f-4bcb-aa4f-adb7bd780159-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-jzz7s\" (UID: \"c9455dd0-779f-4bcb-aa4f-adb7bd780159\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-jzz7s"
Oct 30 21:59:03.091394 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:03.091372    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c9455dd0-779f-4bcb-aa4f-adb7bd780159-workload-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-jzz7s\" (UID: \"c9455dd0-779f-4bcb-aa4f-adb7bd780159\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-jzz7s"
Oct 30 21:59:03.091486 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:03.091423    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c9455dd0-779f-4bcb-aa4f-adb7bd780159-istio-data\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-jzz7s\" (UID: \"c9455dd0-779f-4bcb-aa4f-adb7bd780159\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-jzz7s"
Oct 30 21:59:03.091549 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:03.091502    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c9455dd0-779f-4bcb-aa4f-adb7bd780159-workload-certs\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-jzz7s\" (UID: \"c9455dd0-779f-4bcb-aa4f-adb7bd780159\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-jzz7s"
Oct 30 21:59:03.092621 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:03.092587    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c9455dd0-779f-4bcb-aa4f-adb7bd780159-istio-envoy\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-jzz7s\" (UID: \"c9455dd0-779f-4bcb-aa4f-adb7bd780159\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-jzz7s"
Oct 30 21:59:03.093107 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:03.093080    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c9455dd0-779f-4bcb-aa4f-adb7bd780159-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-jzz7s\" (UID: \"c9455dd0-779f-4bcb-aa4f-adb7bd780159\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-jzz7s"
Oct 30 21:59:03.149249 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:03.149220    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c9455dd0-779f-4bcb-aa4f-adb7bd780159-istio-token\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-jzz7s\" (UID: \"c9455dd0-779f-4bcb-aa4f-adb7bd780159\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-jzz7s"
Oct 30 21:59:03.149433 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:03.149296    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r5pz8\" (UniqueName: \"kubernetes.io/projected/c9455dd0-779f-4bcb-aa4f-adb7bd780159-kube-api-access-r5pz8\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-jzz7s\" (UID: \"c9455dd0-779f-4bcb-aa4f-adb7bd780159\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-jzz7s"
Oct 30 21:59:03.179825 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:03.179792    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-jzz7s"
Oct 30 21:59:03.612383 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:03.612356    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-jzz7s"]
Oct 30 21:59:03.612968 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:59:03.612937    4473 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc9455dd0_779f_4bcb_aa4f_adb7bd780159.slice/crio-b28164a29bb97549e77fc7705739405ed7429a345c75f47053942a133cea573d WatchSource:0}: Error finding container b28164a29bb97549e77fc7705739405ed7429a345c75f47053942a133cea573d: Status 404 returned error can't find the container with id b28164a29bb97549e77fc7705739405ed7429a345c75f47053942a133cea573d
Oct 30 21:59:03.615317 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:03.615280    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:59:03.615402 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:03.615352    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:59:03.615402 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:03.615384    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:59:04.412542 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:04.412505    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-jzz7s" event={"ID":"c9455dd0-779f-4bcb-aa4f-adb7bd780159","Type":"ContainerStarted","Data":"54e8913e351e83423e8649898d06ddc6e926c776babe72c9ad1ef1e9ef213ad8"}
Oct 30 21:59:04.413067 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:04.412555    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-jzz7s" event={"ID":"c9455dd0-779f-4bcb-aa4f-adb7bd780159","Type":"ContainerStarted","Data":"b28164a29bb97549e77fc7705739405ed7429a345c75f47053942a133cea573d"}
Oct 30 21:59:04.721025 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:04.720888    4473 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-jzz7s" podStartSLOduration=2.7208489670000002 podStartE2EDuration="2.720848967s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:59:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:59:04.549683717 +0000 UTC m=+990.323052206" watchObservedRunningTime="2026-10-30 21:59:04.720848967 +0000 UTC m=+990.494217456"
Oct 30 21:59:04.724736 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:04.724701    4473 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w5-9f8aea/distinct-w5-9f8aea-kserve-5b564579cd-kmxzz"]
Oct 30 21:59:04.763162 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:04.763129    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-9f8aea/distinct-w5-9f8aea-kserve-5b564579cd-kmxzz"
Oct 30 21:59:04.791085 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:04.791033    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w5-9f8aea/distinct-w5-9f8aea-kserve-5b564579cd-kmxzz"]
Oct 30 21:59:04.817302 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:04.817261    4473 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w5-9f8aea\"/\"openshift-service-ca.crt\""
Oct 30 21:59:04.817302 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:04.817288    4473 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w5-9f8aea\"/\"kube-root-ca.crt\""
Oct 30 21:59:04.817532 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:04.817346    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w5-9f8aea\"/\"default-dockercfg-vm2j7\""
Oct 30 21:59:04.817532 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:04.817483    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w5-9f8aea\"/\"distinct-w5-9f8aea-kserve-self-signed-certs\""
Oct 30 21:59:04.912153 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:04.911810    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b11e8b11-334f-4a9e-98c2-b0164567a8af-model-cache\") pod \"distinct-w5-9f8aea-kserve-5b564579cd-kmxzz\" (UID: \"b11e8b11-334f-4a9e-98c2-b0164567a8af\") " pod="e2e-models-e2e-worker-w5-9f8aea/distinct-w5-9f8aea-kserve-5b564579cd-kmxzz"
Oct 30 21:59:04.912718 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:04.912523    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7fdfq\" (UniqueName: \"kubernetes.io/projected/b11e8b11-334f-4a9e-98c2-b0164567a8af-kube-api-access-7fdfq\") pod \"distinct-w5-9f8aea-kserve-5b564579cd-kmxzz\" (UID: \"b11e8b11-334f-4a9e-98c2-b0164567a8af\") " pod="e2e-models-e2e-worker-w5-9f8aea/distinct-w5-9f8aea-kserve-5b564579cd-kmxzz"
Oct 30 21:59:04.912955 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:04.912902    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b11e8b11-334f-4a9e-98c2-b0164567a8af-home\") pod \"distinct-w5-9f8aea-kserve-5b564579cd-kmxzz\" (UID: \"b11e8b11-334f-4a9e-98c2-b0164567a8af\") " pod="e2e-models-e2e-worker-w5-9f8aea/distinct-w5-9f8aea-kserve-5b564579cd-kmxzz"
Oct 30 21:59:04.913202 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:04.913161    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b11e8b11-334f-4a9e-98c2-b0164567a8af-dshm\") pod \"distinct-w5-9f8aea-kserve-5b564579cd-kmxzz\" (UID: \"b11e8b11-334f-4a9e-98c2-b0164567a8af\") " pod="e2e-models-e2e-worker-w5-9f8aea/distinct-w5-9f8aea-kserve-5b564579cd-kmxzz"
Oct 30 21:59:04.913573 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:04.913554    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b11e8b11-334f-4a9e-98c2-b0164567a8af-tmp-dir\") pod \"distinct-w5-9f8aea-kserve-5b564579cd-kmxzz\" (UID: \"b11e8b11-334f-4a9e-98c2-b0164567a8af\") " pod="e2e-models-e2e-worker-w5-9f8aea/distinct-w5-9f8aea-kserve-5b564579cd-kmxzz"
Oct 30 21:59:04.913862 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:04.913845    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b11e8b11-334f-4a9e-98c2-b0164567a8af-tls-certs\") pod \"distinct-w5-9f8aea-kserve-5b564579cd-kmxzz\" (UID: \"b11e8b11-334f-4a9e-98c2-b0164567a8af\") " pod="e2e-models-e2e-worker-w5-9f8aea/distinct-w5-9f8aea-kserve-5b564579cd-kmxzz"
Oct 30 21:59:05.015753 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:05.015660    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7fdfq\" (UniqueName: \"kubernetes.io/projected/b11e8b11-334f-4a9e-98c2-b0164567a8af-kube-api-access-7fdfq\") pod \"distinct-w5-9f8aea-kserve-5b564579cd-kmxzz\" (UID: \"b11e8b11-334f-4a9e-98c2-b0164567a8af\") " pod="e2e-models-e2e-worker-w5-9f8aea/distinct-w5-9f8aea-kserve-5b564579cd-kmxzz"
Oct 30 21:59:05.015753 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:05.015700    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b11e8b11-334f-4a9e-98c2-b0164567a8af-home\") pod \"distinct-w5-9f8aea-kserve-5b564579cd-kmxzz\" (UID: \"b11e8b11-334f-4a9e-98c2-b0164567a8af\") " pod="e2e-models-e2e-worker-w5-9f8aea/distinct-w5-9f8aea-kserve-5b564579cd-kmxzz"
Oct 30 21:59:05.015753 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:05.015734    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b11e8b11-334f-4a9e-98c2-b0164567a8af-dshm\") pod \"distinct-w5-9f8aea-kserve-5b564579cd-kmxzz\" (UID: \"b11e8b11-334f-4a9e-98c2-b0164567a8af\") " pod="e2e-models-e2e-worker-w5-9f8aea/distinct-w5-9f8aea-kserve-5b564579cd-kmxzz"
Oct 30 21:59:05.016078 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:05.015762    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b11e8b11-334f-4a9e-98c2-b0164567a8af-tmp-dir\") pod \"distinct-w5-9f8aea-kserve-5b564579cd-kmxzz\" (UID: \"b11e8b11-334f-4a9e-98c2-b0164567a8af\") " pod="e2e-models-e2e-worker-w5-9f8aea/distinct-w5-9f8aea-kserve-5b564579cd-kmxzz"
Oct 30 21:59:05.016078 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:05.015815    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b11e8b11-334f-4a9e-98c2-b0164567a8af-tls-certs\") pod \"distinct-w5-9f8aea-kserve-5b564579cd-kmxzz\" (UID: \"b11e8b11-334f-4a9e-98c2-b0164567a8af\") " pod="e2e-models-e2e-worker-w5-9f8aea/distinct-w5-9f8aea-kserve-5b564579cd-kmxzz"
Oct 30 21:59:05.016078 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:05.015915    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b11e8b11-334f-4a9e-98c2-b0164567a8af-model-cache\") pod \"distinct-w5-9f8aea-kserve-5b564579cd-kmxzz\" (UID: \"b11e8b11-334f-4a9e-98c2-b0164567a8af\") " pod="e2e-models-e2e-worker-w5-9f8aea/distinct-w5-9f8aea-kserve-5b564579cd-kmxzz"
Oct 30 21:59:05.016248 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:05.016169    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b11e8b11-334f-4a9e-98c2-b0164567a8af-home\") pod \"distinct-w5-9f8aea-kserve-5b564579cd-kmxzz\" (UID: \"b11e8b11-334f-4a9e-98c2-b0164567a8af\") " pod="e2e-models-e2e-worker-w5-9f8aea/distinct-w5-9f8aea-kserve-5b564579cd-kmxzz"
Oct 30 21:59:05.016309 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:05.016287    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b11e8b11-334f-4a9e-98c2-b0164567a8af-tmp-dir\") pod \"distinct-w5-9f8aea-kserve-5b564579cd-kmxzz\" (UID: \"b11e8b11-334f-4a9e-98c2-b0164567a8af\") " pod="e2e-models-e2e-worker-w5-9f8aea/distinct-w5-9f8aea-kserve-5b564579cd-kmxzz"
Oct 30 21:59:05.016351 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:05.016332    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b11e8b11-334f-4a9e-98c2-b0164567a8af-model-cache\") pod \"distinct-w5-9f8aea-kserve-5b564579cd-kmxzz\" (UID: \"b11e8b11-334f-4a9e-98c2-b0164567a8af\") " pod="e2e-models-e2e-worker-w5-9f8aea/distinct-w5-9f8aea-kserve-5b564579cd-kmxzz"
Oct 30 21:59:05.018431 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:05.018410    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b11e8b11-334f-4a9e-98c2-b0164567a8af-dshm\") pod \"distinct-w5-9f8aea-kserve-5b564579cd-kmxzz\" (UID: \"b11e8b11-334f-4a9e-98c2-b0164567a8af\") " pod="e2e-models-e2e-worker-w5-9f8aea/distinct-w5-9f8aea-kserve-5b564579cd-kmxzz"
Oct 30 21:59:05.018632 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:05.018618    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b11e8b11-334f-4a9e-98c2-b0164567a8af-tls-certs\") pod \"distinct-w5-9f8aea-kserve-5b564579cd-kmxzz\" (UID: \"b11e8b11-334f-4a9e-98c2-b0164567a8af\") " pod="e2e-models-e2e-worker-w5-9f8aea/distinct-w5-9f8aea-kserve-5b564579cd-kmxzz"
Oct 30 21:59:05.073594 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:05.073559    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7fdfq\" (UniqueName: \"kubernetes.io/projected/b11e8b11-334f-4a9e-98c2-b0164567a8af-kube-api-access-7fdfq\") pod \"distinct-w5-9f8aea-kserve-5b564579cd-kmxzz\" (UID: \"b11e8b11-334f-4a9e-98c2-b0164567a8af\") " pod="e2e-models-e2e-worker-w5-9f8aea/distinct-w5-9f8aea-kserve-5b564579cd-kmxzz"
Oct 30 21:59:05.080199 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:05.080165    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-9f8aea/distinct-w5-9f8aea-kserve-5b564579cd-kmxzz"
Oct 30 21:59:05.180839 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:05.180571    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-jzz7s"
Oct 30 21:59:05.182494 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:05.182255    4473 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-jzz7s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body=
Oct 30 21:59:05.182494 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:05.182313    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-jzz7s" podUID="c9455dd0-779f-4bcb-aa4f-adb7bd780159" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused"
Oct 30 21:59:05.551733 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:05.551655    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w5-9f8aea/distinct-w5-9f8aea-kserve-5b564579cd-kmxzz"]
Oct 30 21:59:05.553307 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:59:05.553274    4473 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb11e8b11_334f_4a9e_98c2_b0164567a8af.slice/crio-fff3bdc0d8e9077354df011d4bae82bbef91e131debca0c78a5848167cb457b3 WatchSource:0}: Error finding container fff3bdc0d8e9077354df011d4bae82bbef91e131debca0c78a5848167cb457b3: Status 404 returned error can't find the container with id fff3bdc0d8e9077354df011d4bae82bbef91e131debca0c78a5848167cb457b3
Oct 30 21:59:06.180899 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:06.180858    4473 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-jzz7s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body=
Oct 30 21:59:06.181104 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:06.180934    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-jzz7s" podUID="c9455dd0-779f-4bcb-aa4f-adb7bd780159" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused"
Oct 30 21:59:06.457184 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:06.457088    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-9f8aea/distinct-w5-9f8aea-kserve-5b564579cd-kmxzz" event={"ID":"b11e8b11-334f-4a9e-98c2-b0164567a8af","Type":"ContainerStarted","Data":"23e7478f0e398bb331ec0933335a676ae4bb1b251a6f49d13a25042b296a8bcb"}
Oct 30 21:59:06.457184 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:06.457135    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-9f8aea/distinct-w5-9f8aea-kserve-5b564579cd-kmxzz" event={"ID":"b11e8b11-334f-4a9e-98c2-b0164567a8af","Type":"ContainerStarted","Data":"fff3bdc0d8e9077354df011d4bae82bbef91e131debca0c78a5848167cb457b3"}
Oct 30 21:59:06.575420 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:06.575367    4473 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w5-9f8aea/distinct-w5-9f8aea-kserve-5b564579cd-kmxzz" podStartSLOduration=2.575352177 podStartE2EDuration="2.575352177s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:59:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:59:06.573864468 +0000 UTC m=+992.347232959" watchObservedRunningTime="2026-10-30 21:59:06.575352177 +0000 UTC m=+992.348720663"
Oct 30 21:59:07.080984 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:07.080943    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-9f8aea/distinct-w5-9f8aea-kserve-5b564579cd-kmxzz"
Oct 30 21:59:07.180578 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:07.180540    4473 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-jzz7s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body=
Oct 30 21:59:07.180758 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:07.180603    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-jzz7s" podUID="c9455dd0-779f-4bcb-aa4f-adb7bd780159" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused"
Oct 30 21:59:08.181145 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:08.181108    4473 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-jzz7s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body=
Oct 30 21:59:08.181558 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:08.181164    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-jzz7s" podUID="c9455dd0-779f-4bcb-aa4f-adb7bd780159" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused"
Oct 30 21:59:09.181043 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:09.181003    4473 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-jzz7s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body=
Oct 30 21:59:09.181256 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:09.181059    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-jzz7s" podUID="c9455dd0-779f-4bcb-aa4f-adb7bd780159" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused"
Oct 30 21:59:10.180622 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:10.180587    4473 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-jzz7s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body=
Oct 30 21:59:10.180819 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:10.180643    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-jzz7s" podUID="c9455dd0-779f-4bcb-aa4f-adb7bd780159" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused"
Oct 30 21:59:11.180333 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:11.180296    4473 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-jzz7s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body=
Oct 30 21:59:11.180743 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:11.180348    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-jzz7s" podUID="c9455dd0-779f-4bcb-aa4f-adb7bd780159" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused"
Oct 30 21:59:12.471824 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:12.470216    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-jzz7s"
Oct 30 21:59:12.471824 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:12.471660    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-jzz7s"
Oct 30 21:59:12.475180 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:12.475150    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-jzz7s"
Oct 30 21:59:15.080607 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:15.080552    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w5-9f8aea/distinct-w5-9f8aea-kserve-5b564579cd-kmxzz"
Oct 30 21:59:15.094799 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:15.094771    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w5-9f8aea/distinct-w5-9f8aea-kserve-5b564579cd-kmxzz"
Oct 30 21:59:15.694755 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:15.694725    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-9f8aea/distinct-w5-9f8aea-kserve-5b564579cd-kmxzz"
Oct 30 21:59:17.868528 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:17.868496    4473 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-wbzjg"]
Oct 30 21:59:17.872578 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:17.872553    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-wbzjg"
Oct 30 21:59:17.982524 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:17.982471    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-wbzjg"]
Oct 30 21:59:18.003247 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:18.003211    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4c4ef828-d895-488c-900d-3525e3fda397-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-wbzjg\" (UID: \"4c4ef828-d895-488c-900d-3525e3fda397\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-wbzjg"
Oct 30 21:59:18.003247 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:18.003259    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4c4ef828-d895-488c-900d-3525e3fda397-workload-certs\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-wbzjg\" (UID: \"4c4ef828-d895-488c-900d-3525e3fda397\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-wbzjg"
Oct 30 21:59:18.003539 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:18.003354    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rp9bs\" (UniqueName: \"kubernetes.io/projected/4c4ef828-d895-488c-900d-3525e3fda397-kube-api-access-rp9bs\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-wbzjg\" (UID: \"4c4ef828-d895-488c-900d-3525e3fda397\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-wbzjg"
Oct 30 21:59:18.003539 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:18.003396    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4c4ef828-d895-488c-900d-3525e3fda397-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-wbzjg\" (UID: \"4c4ef828-d895-488c-900d-3525e3fda397\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-wbzjg"
Oct 30 21:59:18.003539 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:18.003511    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c4ef828-d895-488c-900d-3525e3fda397-workload-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-wbzjg\" (UID: \"4c4ef828-d895-488c-900d-3525e3fda397\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-wbzjg"
Oct 30 21:59:18.003713 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:18.003541    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4c4ef828-d895-488c-900d-3525e3fda397-istio-data\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-wbzjg\" (UID: \"4c4ef828-d895-488c-900d-3525e3fda397\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-wbzjg"
Oct 30 21:59:18.003713 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:18.003561    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4c4ef828-d895-488c-900d-3525e3fda397-istio-envoy\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-wbzjg\" (UID: \"4c4ef828-d895-488c-900d-3525e3fda397\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-wbzjg"
Oct 30 21:59:18.003713 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:18.003589    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4c4ef828-d895-488c-900d-3525e3fda397-istio-token\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-wbzjg\" (UID: \"4c4ef828-d895-488c-900d-3525e3fda397\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-wbzjg"
Oct 30 21:59:18.003713 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:18.003671    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c4ef828-d895-488c-900d-3525e3fda397-credential-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-wbzjg\" (UID: \"4c4ef828-d895-488c-900d-3525e3fda397\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-wbzjg"
Oct 30 21:59:18.104786 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:18.104746    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4c4ef828-d895-488c-900d-3525e3fda397-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-wbzjg\" (UID: \"4c4ef828-d895-488c-900d-3525e3fda397\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-wbzjg"
Oct 30 21:59:18.104786 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:18.104792    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4c4ef828-d895-488c-900d-3525e3fda397-workload-certs\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-wbzjg\" (UID: \"4c4ef828-d895-488c-900d-3525e3fda397\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-wbzjg"
Oct 30 21:59:18.105082 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:18.104826    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rp9bs\" (UniqueName: \"kubernetes.io/projected/4c4ef828-d895-488c-900d-3525e3fda397-kube-api-access-rp9bs\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-wbzjg\" (UID: \"4c4ef828-d895-488c-900d-3525e3fda397\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-wbzjg"
Oct 30 21:59:18.105082 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:18.104848    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4c4ef828-d895-488c-900d-3525e3fda397-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-wbzjg\" (UID: \"4c4ef828-d895-488c-900d-3525e3fda397\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-wbzjg"
Oct 30 21:59:18.105082 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:18.104905    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c4ef828-d895-488c-900d-3525e3fda397-workload-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-wbzjg\" (UID: \"4c4ef828-d895-488c-900d-3525e3fda397\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-wbzjg"
Oct 30 21:59:18.105082 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:18.104946    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4c4ef828-d895-488c-900d-3525e3fda397-istio-data\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-wbzjg\" (UID: \"4c4ef828-d895-488c-900d-3525e3fda397\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-wbzjg"
Oct 30 21:59:18.105290 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:18.105120    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4c4ef828-d895-488c-900d-3525e3fda397-istio-envoy\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-wbzjg\" (UID: \"4c4ef828-d895-488c-900d-3525e3fda397\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-wbzjg"
Oct 30 21:59:18.105290 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:18.105190    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4c4ef828-d895-488c-900d-3525e3fda397-istio-token\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-wbzjg\" (UID: \"4c4ef828-d895-488c-900d-3525e3fda397\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-wbzjg"
Oct 30 21:59:18.105290 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:18.105249    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c4ef828-d895-488c-900d-3525e3fda397-credential-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-wbzjg\" (UID: \"4c4ef828-d895-488c-900d-3525e3fda397\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-wbzjg"
Oct 30 21:59:18.105447 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:18.105347    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4c4ef828-d895-488c-900d-3525e3fda397-workload-certs\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-wbzjg\" (UID: \"4c4ef828-d895-488c-900d-3525e3fda397\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-wbzjg"
Oct 30 21:59:18.105447 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:18.105362    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4c4ef828-d895-488c-900d-3525e3fda397-istio-data\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-wbzjg\" (UID: \"4c4ef828-d895-488c-900d-3525e3fda397\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-wbzjg"
Oct 30 21:59:18.105531 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:18.105468    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c4ef828-d895-488c-900d-3525e3fda397-workload-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-wbzjg\" (UID: \"4c4ef828-d895-488c-900d-3525e3fda397\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-wbzjg"
Oct 30 21:59:18.105639 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:18.105610    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c4ef828-d895-488c-900d-3525e3fda397-credential-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-wbzjg\" (UID: \"4c4ef828-d895-488c-900d-3525e3fda397\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-wbzjg"
Oct 30 21:59:18.105913 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:18.105891    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4c4ef828-d895-488c-900d-3525e3fda397-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-wbzjg\" (UID: \"4c4ef828-d895-488c-900d-3525e3fda397\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-wbzjg"
Oct 30 21:59:18.107481 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:18.107453    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4c4ef828-d895-488c-900d-3525e3fda397-istio-envoy\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-wbzjg\" (UID: \"4c4ef828-d895-488c-900d-3525e3fda397\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-wbzjg"
Oct 30 21:59:18.107561 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:18.107482    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4c4ef828-d895-488c-900d-3525e3fda397-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-wbzjg\" (UID: \"4c4ef828-d895-488c-900d-3525e3fda397\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-wbzjg"
Oct 30 21:59:18.163432 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:18.163360    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4c4ef828-d895-488c-900d-3525e3fda397-istio-token\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-wbzjg\" (UID: \"4c4ef828-d895-488c-900d-3525e3fda397\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-wbzjg"
Oct 30 21:59:18.163567 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:18.163509    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rp9bs\" (UniqueName: \"kubernetes.io/projected/4c4ef828-d895-488c-900d-3525e3fda397-kube-api-access-rp9bs\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-wbzjg\" (UID: \"4c4ef828-d895-488c-900d-3525e3fda397\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-wbzjg"
Oct 30 21:59:18.187203 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:18.187167    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-wbzjg"
Oct 30 21:59:18.672608 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:18.672581    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-wbzjg"]
Oct 30 21:59:18.674399 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:59:18.674374    4473 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c4ef828_d895_488c_900d_3525e3fda397.slice/crio-46b5504379e0e10585bfb5292d5912fb2e3cba5a6a197c5be0d576382baf482a WatchSource:0}: Error finding container 46b5504379e0e10585bfb5292d5912fb2e3cba5a6a197c5be0d576382baf482a: Status 404 returned error can't find the container with id 46b5504379e0e10585bfb5292d5912fb2e3cba5a6a197c5be0d576382baf482a
Oct 30 21:59:18.676659 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:18.676626    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:59:18.676742 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:18.676691    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:59:18.676742 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:18.676720    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:59:18.704944 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:18.704896    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-wbzjg" event={"ID":"4c4ef828-d895-488c-900d-3525e3fda397","Type":"ContainerStarted","Data":"46b5504379e0e10585bfb5292d5912fb2e3cba5a6a197c5be0d576382baf482a"}
Oct 30 21:59:19.712498 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:19.712456    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-wbzjg" event={"ID":"4c4ef828-d895-488c-900d-3525e3fda397","Type":"ContainerStarted","Data":"ec24f45fc0eb542885d4c0e99a28f87054f4b6ec7879bbfb16be4742a91881d0"}
Oct 30 21:59:19.881892 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:19.881837    4473 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-wbzjg" podStartSLOduration=2.881822085 podStartE2EDuration="2.881822085s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:59:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:59:19.880820599 +0000 UTC m=+1005.654189087" watchObservedRunningTime="2026-10-30 21:59:19.881822085 +0000 UTC m=+1005.655190568"
Oct 30 21:59:20.187593 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:20.187558    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-wbzjg"
Oct 30 21:59:20.189153 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:20.189127    4473 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-wbzjg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body=
Oct 30 21:59:20.189294 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:20.189177    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-wbzjg" podUID="4c4ef828-d895-488c-900d-3525e3fda397" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused"
Oct 30 21:59:20.327956 ip-10-0-140-18 kubenswrapper[4473]: E1030 21:59:20.325124    4473 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fncvg" containerName="extractor"
Oct 30 21:59:21.188213 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:21.188170    4473 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-wbzjg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body=
Oct 30 21:59:21.188627 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:21.188242    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-wbzjg" podUID="4c4ef828-d895-488c-900d-3525e3fda397" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused"
Oct 30 21:59:22.188069 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:22.188031    4473 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-wbzjg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body=
Oct 30 21:59:22.188255 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:22.188100    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-wbzjg" podUID="4c4ef828-d895-488c-900d-3525e3fda397" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused"
Oct 30 21:59:23.188294 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:23.188258    4473 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-wbzjg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body=
Oct 30 21:59:23.188701 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:23.188321    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-wbzjg" podUID="4c4ef828-d895-488c-900d-3525e3fda397" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused"
Oct 30 21:59:24.188471 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:24.188428    4473 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-wbzjg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body=
Oct 30 21:59:24.188873 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:24.188496    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-wbzjg" podUID="4c4ef828-d895-488c-900d-3525e3fda397" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused"
Oct 30 21:59:25.188255 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:25.188215    4473 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-wbzjg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body=
Oct 30 21:59:25.188454 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:25.188275    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-wbzjg" podUID="4c4ef828-d895-488c-900d-3525e3fda397" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused"
Oct 30 21:59:26.188402 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:26.188354    4473 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-wbzjg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body=
Oct 30 21:59:26.188818 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:26.188443    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-wbzjg" podUID="4c4ef828-d895-488c-900d-3525e3fda397" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused"
Oct 30 21:59:27.211339 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:27.211290    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-wbzjg"
Oct 30 21:59:27.211845 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:27.211741    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-wbzjg"
Oct 30 21:59:27.212679 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:27.212660    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-wbzjg"
Oct 30 21:59:37.602835 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:37.602793    4473 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-vbr9j"]
Oct 30 21:59:37.607087 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:37.607065    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-vbr9j"
Oct 30 21:59:37.689373 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:37.689314    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b9f9cb51-3fe0-43f3-9be3-95388ed892c8-istio-podinfo\") pod \"e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-vbr9j\" (UID: \"b9f9cb51-3fe0-43f3-9be3-95388ed892c8\") " pod="openshift-ingress/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-vbr9j"
Oct 30 21:59:37.689373 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:37.689385    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b9f9cb51-3fe0-43f3-9be3-95388ed892c8-workload-certs\") pod \"e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-vbr9j\" (UID: \"b9f9cb51-3fe0-43f3-9be3-95388ed892c8\") " pod="openshift-ingress/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-vbr9j"
Oct 30 21:59:37.689666 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:37.689519    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b9f9cb51-3fe0-43f3-9be3-95388ed892c8-workload-socket\") pod \"e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-vbr9j\" (UID: \"b9f9cb51-3fe0-43f3-9be3-95388ed892c8\") " pod="openshift-ingress/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-vbr9j"
Oct 30 21:59:37.689745 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:37.689696    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b9f9cb51-3fe0-43f3-9be3-95388ed892c8-istio-envoy\") pod \"e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-vbr9j\" (UID: \"b9f9cb51-3fe0-43f3-9be3-95388ed892c8\") " pod="openshift-ingress/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-vbr9j"
Oct 30 21:59:37.689804 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:37.689755    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b9f9cb51-3fe0-43f3-9be3-95388ed892c8-credential-socket\") pod \"e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-vbr9j\" (UID: \"b9f9cb51-3fe0-43f3-9be3-95388ed892c8\") " pod="openshift-ingress/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-vbr9j"
Oct 30 21:59:37.689911 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:37.689883    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b9f9cb51-3fe0-43f3-9be3-95388ed892c8-istiod-ca-cert\") pod \"e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-vbr9j\" (UID: \"b9f9cb51-3fe0-43f3-9be3-95388ed892c8\") " pod="openshift-ingress/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-vbr9j"
Oct 30 21:59:37.690036 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:37.689953    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r2gjp\" (UniqueName: \"kubernetes.io/projected/b9f9cb51-3fe0-43f3-9be3-95388ed892c8-kube-api-access-r2gjp\") pod \"e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-vbr9j\" (UID: \"b9f9cb51-3fe0-43f3-9be3-95388ed892c8\") " pod="openshift-ingress/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-vbr9j"
Oct 30 21:59:37.690103 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:37.690083    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b9f9cb51-3fe0-43f3-9be3-95388ed892c8-istio-token\") pod \"e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-vbr9j\" (UID: \"b9f9cb51-3fe0-43f3-9be3-95388ed892c8\") " pod="openshift-ingress/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-vbr9j"
Oct 30 21:59:37.690163 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:37.690119    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b9f9cb51-3fe0-43f3-9be3-95388ed892c8-istio-data\") pod \"e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-vbr9j\" (UID: \"b9f9cb51-3fe0-43f3-9be3-95388ed892c8\") " pod="openshift-ingress/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-vbr9j"
Oct 30 21:59:37.694511 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:37.694478    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-vbr9j"]
Oct 30 21:59:37.791756 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:37.791710    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b9f9cb51-3fe0-43f3-9be3-95388ed892c8-istio-token\") pod \"e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-vbr9j\" (UID: \"b9f9cb51-3fe0-43f3-9be3-95388ed892c8\") " pod="openshift-ingress/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-vbr9j"
Oct 30 21:59:37.791756 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:37.791761    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b9f9cb51-3fe0-43f3-9be3-95388ed892c8-istio-data\") pod \"e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-vbr9j\" (UID: \"b9f9cb51-3fe0-43f3-9be3-95388ed892c8\") " pod="openshift-ingress/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-vbr9j"
Oct 30 21:59:37.792024 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:37.791832    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b9f9cb51-3fe0-43f3-9be3-95388ed892c8-istio-podinfo\") pod \"e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-vbr9j\" (UID: \"b9f9cb51-3fe0-43f3-9be3-95388ed892c8\") " pod="openshift-ingress/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-vbr9j"
Oct 30 21:59:37.792024 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:37.791866    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b9f9cb51-3fe0-43f3-9be3-95388ed892c8-workload-certs\") pod \"e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-vbr9j\" (UID: \"b9f9cb51-3fe0-43f3-9be3-95388ed892c8\") " pod="openshift-ingress/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-vbr9j"
Oct 30 21:59:37.792024 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:37.791974    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b9f9cb51-3fe0-43f3-9be3-95388ed892c8-workload-socket\") pod \"e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-vbr9j\" (UID: \"b9f9cb51-3fe0-43f3-9be3-95388ed892c8\") " pod="openshift-ingress/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-vbr9j"
Oct 30 21:59:37.792183 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:37.792032    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b9f9cb51-3fe0-43f3-9be3-95388ed892c8-istio-envoy\") pod \"e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-vbr9j\" (UID: \"b9f9cb51-3fe0-43f3-9be3-95388ed892c8\") " pod="openshift-ingress/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-vbr9j"
Oct 30 21:59:37.792183 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:37.792081    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b9f9cb51-3fe0-43f3-9be3-95388ed892c8-credential-socket\") pod \"e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-vbr9j\" (UID: \"b9f9cb51-3fe0-43f3-9be3-95388ed892c8\") " pod="openshift-ingress/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-vbr9j"
Oct 30 21:59:37.792183 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:37.792136    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b9f9cb51-3fe0-43f3-9be3-95388ed892c8-istiod-ca-cert\") pod \"e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-vbr9j\" (UID: \"b9f9cb51-3fe0-43f3-9be3-95388ed892c8\") " pod="openshift-ingress/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-vbr9j"
Oct 30 21:59:37.792183 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:37.792171    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r2gjp\" (UniqueName: \"kubernetes.io/projected/b9f9cb51-3fe0-43f3-9be3-95388ed892c8-kube-api-access-r2gjp\") pod \"e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-vbr9j\" (UID: \"b9f9cb51-3fe0-43f3-9be3-95388ed892c8\") " pod="openshift-ingress/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-vbr9j"
Oct 30 21:59:37.792390 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:37.792282    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b9f9cb51-3fe0-43f3-9be3-95388ed892c8-istio-data\") pod \"e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-vbr9j\" (UID: \"b9f9cb51-3fe0-43f3-9be3-95388ed892c8\") " pod="openshift-ingress/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-vbr9j"
Oct 30 21:59:37.792390 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:37.792337    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b9f9cb51-3fe0-43f3-9be3-95388ed892c8-workload-certs\") pod \"e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-vbr9j\" (UID: \"b9f9cb51-3fe0-43f3-9be3-95388ed892c8\") " pod="openshift-ingress/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-vbr9j"
Oct 30 21:59:37.792390 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:37.792369    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b9f9cb51-3fe0-43f3-9be3-95388ed892c8-workload-socket\") pod \"e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-vbr9j\" (UID: \"b9f9cb51-3fe0-43f3-9be3-95388ed892c8\") " pod="openshift-ingress/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-vbr9j"
Oct 30 21:59:37.792520 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:37.792504    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b9f9cb51-3fe0-43f3-9be3-95388ed892c8-credential-socket\") pod \"e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-vbr9j\" (UID: \"b9f9cb51-3fe0-43f3-9be3-95388ed892c8\") " pod="openshift-ingress/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-vbr9j"
Oct 30 21:59:37.792822 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:37.792799    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b9f9cb51-3fe0-43f3-9be3-95388ed892c8-istiod-ca-cert\") pod \"e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-vbr9j\" (UID: \"b9f9cb51-3fe0-43f3-9be3-95388ed892c8\") " pod="openshift-ingress/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-vbr9j"
Oct 30 21:59:37.794607 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:37.794582    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b9f9cb51-3fe0-43f3-9be3-95388ed892c8-istio-envoy\") pod \"e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-vbr9j\" (UID: \"b9f9cb51-3fe0-43f3-9be3-95388ed892c8\") " pod="openshift-ingress/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-vbr9j"
Oct 30 21:59:37.794758 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:37.794739    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b9f9cb51-3fe0-43f3-9be3-95388ed892c8-istio-podinfo\") pod \"e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-vbr9j\" (UID: \"b9f9cb51-3fe0-43f3-9be3-95388ed892c8\") " pod="openshift-ingress/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-vbr9j"
Oct 30 21:59:37.829890 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:37.829852    4473 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-2lzcz"]
Oct 30 21:59:37.834260 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:37.834233    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-2lzcz"
Oct 30 21:59:37.852476 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:37.852440    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b9f9cb51-3fe0-43f3-9be3-95388ed892c8-istio-token\") pod \"e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-vbr9j\" (UID: \"b9f9cb51-3fe0-43f3-9be3-95388ed892c8\") " pod="openshift-ingress/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-vbr9j"
Oct 30 21:59:37.852657 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:37.852516    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r2gjp\" (UniqueName: \"kubernetes.io/projected/b9f9cb51-3fe0-43f3-9be3-95388ed892c8-kube-api-access-r2gjp\") pod \"e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-vbr9j\" (UID: \"b9f9cb51-3fe0-43f3-9be3-95388ed892c8\") " pod="openshift-ingress/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-vbr9j"
Oct 30 21:59:37.894361 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:37.893997    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/83504be1-81ee-4a97-93e3-a61ecb2f3b2d-istio-token\") pod \"e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-2lzcz\" (UID: \"83504be1-81ee-4a97-93e3-a61ecb2f3b2d\") " pod="openshift-ingress/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-2lzcz"
Oct 30 21:59:37.894361 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:37.894044    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/83504be1-81ee-4a97-93e3-a61ecb2f3b2d-istiod-ca-cert\") pod \"e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-2lzcz\" (UID: \"83504be1-81ee-4a97-93e3-a61ecb2f3b2d\") " pod="openshift-ingress/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-2lzcz"
Oct 30 21:59:37.894361 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:37.894150    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/83504be1-81ee-4a97-93e3-a61ecb2f3b2d-istio-envoy\") pod \"e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-2lzcz\" (UID: \"83504be1-81ee-4a97-93e3-a61ecb2f3b2d\") " pod="openshift-ingress/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-2lzcz"
Oct 30 21:59:37.894361 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:37.894191    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/83504be1-81ee-4a97-93e3-a61ecb2f3b2d-istio-podinfo\") pod \"e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-2lzcz\" (UID: \"83504be1-81ee-4a97-93e3-a61ecb2f3b2d\") " pod="openshift-ingress/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-2lzcz"
Oct 30 21:59:37.894361 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:37.894216    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/83504be1-81ee-4a97-93e3-a61ecb2f3b2d-istio-data\") pod \"e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-2lzcz\" (UID: \"83504be1-81ee-4a97-93e3-a61ecb2f3b2d\") " pod="openshift-ingress/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-2lzcz"
Oct 30 21:59:37.894361 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:37.894273    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-2lzcz"]
Oct 30 21:59:37.894361 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:37.894306    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6869g\" (UniqueName: \"kubernetes.io/projected/83504be1-81ee-4a97-93e3-a61ecb2f3b2d-kube-api-access-6869g\") pod \"e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-2lzcz\" (UID: \"83504be1-81ee-4a97-93e3-a61ecb2f3b2d\") " pod="openshift-ingress/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-2lzcz"
Oct 30 21:59:37.894361 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:37.894377    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/83504be1-81ee-4a97-93e3-a61ecb2f3b2d-credential-socket\") pod \"e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-2lzcz\" (UID: \"83504be1-81ee-4a97-93e3-a61ecb2f3b2d\") " pod="openshift-ingress/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-2lzcz"
Oct 30 21:59:37.894861 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:37.894453    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/83504be1-81ee-4a97-93e3-a61ecb2f3b2d-workload-certs\") pod \"e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-2lzcz\" (UID: \"83504be1-81ee-4a97-93e3-a61ecb2f3b2d\") " pod="openshift-ingress/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-2lzcz"
Oct 30 21:59:37.894861 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:37.894515    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/83504be1-81ee-4a97-93e3-a61ecb2f3b2d-workload-socket\") pod \"e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-2lzcz\" (UID: \"83504be1-81ee-4a97-93e3-a61ecb2f3b2d\") " pod="openshift-ingress/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-2lzcz"
Oct 30 21:59:37.921654 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:37.921606    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-vbr9j"
Oct 30 21:59:37.999573 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:37.996394    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/83504be1-81ee-4a97-93e3-a61ecb2f3b2d-credential-socket\") pod \"e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-2lzcz\" (UID: \"83504be1-81ee-4a97-93e3-a61ecb2f3b2d\") " pod="openshift-ingress/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-2lzcz"
Oct 30 21:59:37.999573 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:37.996470    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/83504be1-81ee-4a97-93e3-a61ecb2f3b2d-workload-certs\") pod \"e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-2lzcz\" (UID: \"83504be1-81ee-4a97-93e3-a61ecb2f3b2d\") " pod="openshift-ingress/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-2lzcz"
Oct 30 21:59:37.999573 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:37.996567    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/83504be1-81ee-4a97-93e3-a61ecb2f3b2d-workload-socket\") pod \"e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-2lzcz\" (UID: \"83504be1-81ee-4a97-93e3-a61ecb2f3b2d\") " pod="openshift-ingress/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-2lzcz"
Oct 30 21:59:37.999573 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:37.996644    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/83504be1-81ee-4a97-93e3-a61ecb2f3b2d-istio-token\") pod \"e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-2lzcz\" (UID: \"83504be1-81ee-4a97-93e3-a61ecb2f3b2d\") " pod="openshift-ingress/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-2lzcz"
Oct 30 21:59:37.999573 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:37.996679    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/83504be1-81ee-4a97-93e3-a61ecb2f3b2d-istiod-ca-cert\") pod \"e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-2lzcz\" (UID: \"83504be1-81ee-4a97-93e3-a61ecb2f3b2d\") " pod="openshift-ingress/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-2lzcz"
Oct 30 21:59:37.999573 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:37.996748    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/83504be1-81ee-4a97-93e3-a61ecb2f3b2d-istio-envoy\") pod \"e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-2lzcz\" (UID: \"83504be1-81ee-4a97-93e3-a61ecb2f3b2d\") " pod="openshift-ingress/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-2lzcz"
Oct 30 21:59:37.999573 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:37.996780    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/83504be1-81ee-4a97-93e3-a61ecb2f3b2d-istio-podinfo\") pod \"e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-2lzcz\" (UID: \"83504be1-81ee-4a97-93e3-a61ecb2f3b2d\") " pod="openshift-ingress/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-2lzcz"
Oct 30 21:59:37.999573 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:37.996812    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/83504be1-81ee-4a97-93e3-a61ecb2f3b2d-istio-data\") pod \"e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-2lzcz\" (UID: \"83504be1-81ee-4a97-93e3-a61ecb2f3b2d\") " pod="openshift-ingress/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-2lzcz"
Oct 30 21:59:37.999573 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:37.996846    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6869g\" (UniqueName: \"kubernetes.io/projected/83504be1-81ee-4a97-93e3-a61ecb2f3b2d-kube-api-access-6869g\") pod \"e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-2lzcz\" (UID: \"83504be1-81ee-4a97-93e3-a61ecb2f3b2d\") " pod="openshift-ingress/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-2lzcz"
Oct 30 21:59:37.999573 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:37.997675    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/83504be1-81ee-4a97-93e3-a61ecb2f3b2d-credential-socket\") pod \"e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-2lzcz\" (UID: \"83504be1-81ee-4a97-93e3-a61ecb2f3b2d\") " pod="openshift-ingress/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-2lzcz"
Oct 30 21:59:37.999573 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:37.997956    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/83504be1-81ee-4a97-93e3-a61ecb2f3b2d-workload-certs\") pod \"e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-2lzcz\" (UID: \"83504be1-81ee-4a97-93e3-a61ecb2f3b2d\") " pod="openshift-ingress/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-2lzcz"
Oct 30 21:59:37.999573 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:37.998209    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/83504be1-81ee-4a97-93e3-a61ecb2f3b2d-workload-socket\") pod \"e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-2lzcz\" (UID: \"83504be1-81ee-4a97-93e3-a61ecb2f3b2d\") " pod="openshift-ingress/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-2lzcz"
Oct 30 21:59:37.999573 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:37.999248    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/83504be1-81ee-4a97-93e3-a61ecb2f3b2d-istiod-ca-cert\") pod \"e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-2lzcz\" (UID: \"83504be1-81ee-4a97-93e3-a61ecb2f3b2d\") " pod="openshift-ingress/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-2lzcz"
Oct 30 21:59:37.999573 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:37.999289    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/83504be1-81ee-4a97-93e3-a61ecb2f3b2d-istio-data\") pod \"e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-2lzcz\" (UID: \"83504be1-81ee-4a97-93e3-a61ecb2f3b2d\") " pod="openshift-ingress/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-2lzcz"
Oct 30 21:59:38.002265 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:38.002196    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/83504be1-81ee-4a97-93e3-a61ecb2f3b2d-istio-podinfo\") pod \"e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-2lzcz\" (UID: \"83504be1-81ee-4a97-93e3-a61ecb2f3b2d\") " pod="openshift-ingress/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-2lzcz"
Oct 30 21:59:38.003370 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:38.003336    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/83504be1-81ee-4a97-93e3-a61ecb2f3b2d-istio-envoy\") pod \"e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-2lzcz\" (UID: \"83504be1-81ee-4a97-93e3-a61ecb2f3b2d\") " pod="openshift-ingress/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-2lzcz"
Oct 30 21:59:38.056164 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:38.056127    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/83504be1-81ee-4a97-93e3-a61ecb2f3b2d-istio-token\") pod \"e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-2lzcz\" (UID: \"83504be1-81ee-4a97-93e3-a61ecb2f3b2d\") " pod="openshift-ingress/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-2lzcz"
Oct 30 21:59:38.056316 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:38.056188    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6869g\" (UniqueName: \"kubernetes.io/projected/83504be1-81ee-4a97-93e3-a61ecb2f3b2d-kube-api-access-6869g\") pod \"e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-2lzcz\" (UID: \"83504be1-81ee-4a97-93e3-a61ecb2f3b2d\") " pod="openshift-ingress/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-2lzcz"
Oct 30 21:59:38.150684 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:38.150594    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-2lzcz"
Oct 30 21:59:38.193100 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:38.191143    4473 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-chw2v"]
Oct 30 21:59:38.196534 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:38.196500    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-chw2v"
Oct 30 21:59:38.255984 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:38.255945    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-chw2v"]
Oct 30 21:59:38.299790 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:38.299754    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/65b0ee0a-4761-40f3-bd4d-1b20dc75e233-workload-socket\") pod \"e2e-worker-w6-29cbad-openshift-default-6d74d46b79-chw2v\" (UID: \"65b0ee0a-4761-40f3-bd4d-1b20dc75e233\") " pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-chw2v"
Oct 30 21:59:38.299790 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:38.299793    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/65b0ee0a-4761-40f3-bd4d-1b20dc75e233-istio-envoy\") pod \"e2e-worker-w6-29cbad-openshift-default-6d74d46b79-chw2v\" (UID: \"65b0ee0a-4761-40f3-bd4d-1b20dc75e233\") " pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-chw2v"
Oct 30 21:59:38.300050 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:38.299813    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/65b0ee0a-4761-40f3-bd4d-1b20dc75e233-istio-token\") pod \"e2e-worker-w6-29cbad-openshift-default-6d74d46b79-chw2v\" (UID: \"65b0ee0a-4761-40f3-bd4d-1b20dc75e233\") " pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-chw2v"
Oct 30 21:59:38.300050 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:38.299911    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/65b0ee0a-4761-40f3-bd4d-1b20dc75e233-istio-podinfo\") pod \"e2e-worker-w6-29cbad-openshift-default-6d74d46b79-chw2v\" (UID: \"65b0ee0a-4761-40f3-bd4d-1b20dc75e233\") " pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-chw2v"
Oct 30 21:59:38.300050 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:38.300037    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/65b0ee0a-4761-40f3-bd4d-1b20dc75e233-istio-data\") pod \"e2e-worker-w6-29cbad-openshift-default-6d74d46b79-chw2v\" (UID: \"65b0ee0a-4761-40f3-bd4d-1b20dc75e233\") " pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-chw2v"
Oct 30 21:59:38.300150 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:38.300078    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s4xtt\" (UniqueName: \"kubernetes.io/projected/65b0ee0a-4761-40f3-bd4d-1b20dc75e233-kube-api-access-s4xtt\") pod \"e2e-worker-w6-29cbad-openshift-default-6d74d46b79-chw2v\" (UID: \"65b0ee0a-4761-40f3-bd4d-1b20dc75e233\") " pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-chw2v"
Oct 30 21:59:38.300184 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:38.300163    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/65b0ee0a-4761-40f3-bd4d-1b20dc75e233-istiod-ca-cert\") pod \"e2e-worker-w6-29cbad-openshift-default-6d74d46b79-chw2v\" (UID: \"65b0ee0a-4761-40f3-bd4d-1b20dc75e233\") " pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-chw2v"
Oct 30 21:59:38.300231 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:38.300215    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/65b0ee0a-4761-40f3-bd4d-1b20dc75e233-workload-certs\") pod \"e2e-worker-w6-29cbad-openshift-default-6d74d46b79-chw2v\" (UID: \"65b0ee0a-4761-40f3-bd4d-1b20dc75e233\") " pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-chw2v"
Oct 30 21:59:38.300304 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:38.300284    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/65b0ee0a-4761-40f3-bd4d-1b20dc75e233-credential-socket\") pod \"e2e-worker-w6-29cbad-openshift-default-6d74d46b79-chw2v\" (UID: \"65b0ee0a-4761-40f3-bd4d-1b20dc75e233\") " pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-chw2v"
Oct 30 21:59:38.355848 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:59:38.355819    4473 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb9f9cb51_3fe0_43f3_9be3_95388ed892c8.slice/crio-e318c54d504541c8b537b6697d3fb2f924bfc6a0ec9ee0cc30f673f4451915b4 WatchSource:0}: Error finding container e318c54d504541c8b537b6697d3fb2f924bfc6a0ec9ee0cc30f673f4451915b4: Status 404 returned error can't find the container with id e318c54d504541c8b537b6697d3fb2f924bfc6a0ec9ee0cc30f673f4451915b4
Oct 30 21:59:38.363887 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:38.363856    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-vbr9j"]
Oct 30 21:59:38.366147 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:38.366111    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:59:38.366219 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:38.366180    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:59:38.366219 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:38.366209    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:59:38.401252 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:38.401186    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s4xtt\" (UniqueName: \"kubernetes.io/projected/65b0ee0a-4761-40f3-bd4d-1b20dc75e233-kube-api-access-s4xtt\") pod \"e2e-worker-w6-29cbad-openshift-default-6d74d46b79-chw2v\" (UID: \"65b0ee0a-4761-40f3-bd4d-1b20dc75e233\") " pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-chw2v"
Oct 30 21:59:38.401379 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:38.401284    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/65b0ee0a-4761-40f3-bd4d-1b20dc75e233-istiod-ca-cert\") pod \"e2e-worker-w6-29cbad-openshift-default-6d74d46b79-chw2v\" (UID: \"65b0ee0a-4761-40f3-bd4d-1b20dc75e233\") " pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-chw2v"
Oct 30 21:59:38.401379 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:38.401337    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/65b0ee0a-4761-40f3-bd4d-1b20dc75e233-workload-certs\") pod \"e2e-worker-w6-29cbad-openshift-default-6d74d46b79-chw2v\" (UID: \"65b0ee0a-4761-40f3-bd4d-1b20dc75e233\") " pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-chw2v"
Oct 30 21:59:38.401379 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:38.401376    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/65b0ee0a-4761-40f3-bd4d-1b20dc75e233-credential-socket\") pod \"e2e-worker-w6-29cbad-openshift-default-6d74d46b79-chw2v\" (UID: \"65b0ee0a-4761-40f3-bd4d-1b20dc75e233\") " pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-chw2v"
Oct 30 21:59:38.401560 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:38.401539    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/65b0ee0a-4761-40f3-bd4d-1b20dc75e233-workload-socket\") pod \"e2e-worker-w6-29cbad-openshift-default-6d74d46b79-chw2v\" (UID: \"65b0ee0a-4761-40f3-bd4d-1b20dc75e233\") " pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-chw2v"
Oct 30 21:59:38.401613 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:38.401572    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/65b0ee0a-4761-40f3-bd4d-1b20dc75e233-istio-envoy\") pod \"e2e-worker-w6-29cbad-openshift-default-6d74d46b79-chw2v\" (UID: \"65b0ee0a-4761-40f3-bd4d-1b20dc75e233\") " pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-chw2v"
Oct 30 21:59:38.401613 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:38.401601    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/65b0ee0a-4761-40f3-bd4d-1b20dc75e233-istio-token\") pod \"e2e-worker-w6-29cbad-openshift-default-6d74d46b79-chw2v\" (UID: \"65b0ee0a-4761-40f3-bd4d-1b20dc75e233\") " pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-chw2v"
Oct 30 21:59:38.401715 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:38.401663    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/65b0ee0a-4761-40f3-bd4d-1b20dc75e233-istio-podinfo\") pod \"e2e-worker-w6-29cbad-openshift-default-6d74d46b79-chw2v\" (UID: \"65b0ee0a-4761-40f3-bd4d-1b20dc75e233\") " pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-chw2v"
Oct 30 21:59:38.402162 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:38.402138    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/65b0ee0a-4761-40f3-bd4d-1b20dc75e233-istio-data\") pod \"e2e-worker-w6-29cbad-openshift-default-6d74d46b79-chw2v\" (UID: \"65b0ee0a-4761-40f3-bd4d-1b20dc75e233\") " pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-chw2v"
Oct 30 21:59:38.402349 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:38.402321    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/65b0ee0a-4761-40f3-bd4d-1b20dc75e233-workload-socket\") pod \"e2e-worker-w6-29cbad-openshift-default-6d74d46b79-chw2v\" (UID: \"65b0ee0a-4761-40f3-bd4d-1b20dc75e233\") " pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-chw2v"
Oct 30 21:59:38.402426 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:38.402232    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/65b0ee0a-4761-40f3-bd4d-1b20dc75e233-credential-socket\") pod \"e2e-worker-w6-29cbad-openshift-default-6d74d46b79-chw2v\" (UID: \"65b0ee0a-4761-40f3-bd4d-1b20dc75e233\") " pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-chw2v"
Oct 30 21:59:38.402474 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:38.402194    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/65b0ee0a-4761-40f3-bd4d-1b20dc75e233-workload-certs\") pod \"e2e-worker-w6-29cbad-openshift-default-6d74d46b79-chw2v\" (UID: \"65b0ee0a-4761-40f3-bd4d-1b20dc75e233\") " pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-chw2v"
Oct 30 21:59:38.402522 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:38.402485    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/65b0ee0a-4761-40f3-bd4d-1b20dc75e233-istiod-ca-cert\") pod \"e2e-worker-w6-29cbad-openshift-default-6d74d46b79-chw2v\" (UID: \"65b0ee0a-4761-40f3-bd4d-1b20dc75e233\") " pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-chw2v"
Oct 30 21:59:38.402836 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:38.402811    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/65b0ee0a-4761-40f3-bd4d-1b20dc75e233-istio-data\") pod \"e2e-worker-w6-29cbad-openshift-default-6d74d46b79-chw2v\" (UID: \"65b0ee0a-4761-40f3-bd4d-1b20dc75e233\") " pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-chw2v"
Oct 30 21:59:38.404452 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:38.404428    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/65b0ee0a-4761-40f3-bd4d-1b20dc75e233-istio-podinfo\") pod \"e2e-worker-w6-29cbad-openshift-default-6d74d46b79-chw2v\" (UID: \"65b0ee0a-4761-40f3-bd4d-1b20dc75e233\") " pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-chw2v"
Oct 30 21:59:38.404676 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:38.404660    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/65b0ee0a-4761-40f3-bd4d-1b20dc75e233-istio-envoy\") pod \"e2e-worker-w6-29cbad-openshift-default-6d74d46b79-chw2v\" (UID: \"65b0ee0a-4761-40f3-bd4d-1b20dc75e233\") " pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-chw2v"
Oct 30 21:59:38.459964 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:38.459900    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/65b0ee0a-4761-40f3-bd4d-1b20dc75e233-istio-token\") pod \"e2e-worker-w6-29cbad-openshift-default-6d74d46b79-chw2v\" (UID: \"65b0ee0a-4761-40f3-bd4d-1b20dc75e233\") " pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-chw2v"
Oct 30 21:59:38.460129 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:38.460090    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s4xtt\" (UniqueName: \"kubernetes.io/projected/65b0ee0a-4761-40f3-bd4d-1b20dc75e233-kube-api-access-s4xtt\") pod \"e2e-worker-w6-29cbad-openshift-default-6d74d46b79-chw2v\" (UID: \"65b0ee0a-4761-40f3-bd4d-1b20dc75e233\") " pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-chw2v"
Oct 30 21:59:38.515163 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:38.515128    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-chw2v"
Oct 30 21:59:38.539911 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:59:38.539872    4473 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod83504be1_81ee_4a97_93e3_a61ecb2f3b2d.slice/crio-54da5642fe8c061b3b5372de248e911939b297eccb115b71903371ed0069a407 WatchSource:0}: Error finding container 54da5642fe8c061b3b5372de248e911939b297eccb115b71903371ed0069a407: Status 404 returned error can't find the container with id 54da5642fe8c061b3b5372de248e911939b297eccb115b71903371ed0069a407
Oct 30 21:59:38.541415 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:38.541383    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-2lzcz"]
Oct 30 21:59:38.546848 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:38.546807    4473 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-tjjpc"]
Oct 30 21:59:38.549453 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:38.549405    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:59:38.549600 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:38.549497    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:59:38.549600 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:38.549544    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:59:38.552760 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:38.552737    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-tjjpc"
Oct 30 21:59:38.607293 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:38.605785    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a3128eb9-83aa-4ea0-abaf-c642b921870a-workload-certs\") pod \"e2e-worker-w1-d516c4-openshift-default-6bd584745-tjjpc\" (UID: \"a3128eb9-83aa-4ea0-abaf-c642b921870a\") " pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-tjjpc"
Oct 30 21:59:38.607293 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:38.605934    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a3128eb9-83aa-4ea0-abaf-c642b921870a-istio-envoy\") pod \"e2e-worker-w1-d516c4-openshift-default-6bd584745-tjjpc\" (UID: \"a3128eb9-83aa-4ea0-abaf-c642b921870a\") " pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-tjjpc"
Oct 30 21:59:38.607293 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:38.606001    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a3128eb9-83aa-4ea0-abaf-c642b921870a-istiod-ca-cert\") pod \"e2e-worker-w1-d516c4-openshift-default-6bd584745-tjjpc\" (UID: \"a3128eb9-83aa-4ea0-abaf-c642b921870a\") " pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-tjjpc"
Oct 30 21:59:38.609591 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:38.608065    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a3128eb9-83aa-4ea0-abaf-c642b921870a-istio-token\") pod \"e2e-worker-w1-d516c4-openshift-default-6bd584745-tjjpc\" (UID: \"a3128eb9-83aa-4ea0-abaf-c642b921870a\") " pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-tjjpc"
Oct 30 21:59:38.609591 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:38.608143    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3128eb9-83aa-4ea0-abaf-c642b921870a-workload-socket\") pod \"e2e-worker-w1-d516c4-openshift-default-6bd584745-tjjpc\" (UID: \"a3128eb9-83aa-4ea0-abaf-c642b921870a\") " pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-tjjpc"
Oct 30 21:59:38.609591 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:38.608169    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vj7qj\" (UniqueName: \"kubernetes.io/projected/a3128eb9-83aa-4ea0-abaf-c642b921870a-kube-api-access-vj7qj\") pod \"e2e-worker-w1-d516c4-openshift-default-6bd584745-tjjpc\" (UID: \"a3128eb9-83aa-4ea0-abaf-c642b921870a\") " pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-tjjpc"
Oct 30 21:59:38.609591 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:38.608220    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3128eb9-83aa-4ea0-abaf-c642b921870a-credential-socket\") pod \"e2e-worker-w1-d516c4-openshift-default-6bd584745-tjjpc\" (UID: \"a3128eb9-83aa-4ea0-abaf-c642b921870a\") " pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-tjjpc"
Oct 30 21:59:38.609591 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:38.608257    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a3128eb9-83aa-4ea0-abaf-c642b921870a-istio-data\") pod \"e2e-worker-w1-d516c4-openshift-default-6bd584745-tjjpc\" (UID: \"a3128eb9-83aa-4ea0-abaf-c642b921870a\") " pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-tjjpc"
Oct 30 21:59:38.609591 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:38.608311    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a3128eb9-83aa-4ea0-abaf-c642b921870a-istio-podinfo\") pod \"e2e-worker-w1-d516c4-openshift-default-6bd584745-tjjpc\" (UID: \"a3128eb9-83aa-4ea0-abaf-c642b921870a\") " pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-tjjpc"
Oct 30 21:59:38.623807 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:38.623770    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-tjjpc"]
Oct 30 21:59:38.710398 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:38.710310    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a3128eb9-83aa-4ea0-abaf-c642b921870a-workload-certs\") pod \"e2e-worker-w1-d516c4-openshift-default-6bd584745-tjjpc\" (UID: \"a3128eb9-83aa-4ea0-abaf-c642b921870a\") " pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-tjjpc"
Oct 30 21:59:38.710554 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:38.710454    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a3128eb9-83aa-4ea0-abaf-c642b921870a-istio-envoy\") pod \"e2e-worker-w1-d516c4-openshift-default-6bd584745-tjjpc\" (UID: \"a3128eb9-83aa-4ea0-abaf-c642b921870a\") " pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-tjjpc"
Oct 30 21:59:38.710554 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:38.710515    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a3128eb9-83aa-4ea0-abaf-c642b921870a-istiod-ca-cert\") pod \"e2e-worker-w1-d516c4-openshift-default-6bd584745-tjjpc\" (UID: \"a3128eb9-83aa-4ea0-abaf-c642b921870a\") " pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-tjjpc"
Oct 30 21:59:38.710658 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:38.710575    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a3128eb9-83aa-4ea0-abaf-c642b921870a-istio-token\") pod \"e2e-worker-w1-d516c4-openshift-default-6bd584745-tjjpc\" (UID: \"a3128eb9-83aa-4ea0-abaf-c642b921870a\") " pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-tjjpc"
Oct 30 21:59:38.710821 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:38.710781    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a3128eb9-83aa-4ea0-abaf-c642b921870a-workload-certs\") pod \"e2e-worker-w1-d516c4-openshift-default-6bd584745-tjjpc\" (UID: \"a3128eb9-83aa-4ea0-abaf-c642b921870a\") " pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-tjjpc"
Oct 30 21:59:38.711694 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:38.711143    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3128eb9-83aa-4ea0-abaf-c642b921870a-workload-socket\") pod \"e2e-worker-w1-d516c4-openshift-default-6bd584745-tjjpc\" (UID: \"a3128eb9-83aa-4ea0-abaf-c642b921870a\") " pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-tjjpc"
Oct 30 21:59:38.711694 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:38.711200    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vj7qj\" (UniqueName: \"kubernetes.io/projected/a3128eb9-83aa-4ea0-abaf-c642b921870a-kube-api-access-vj7qj\") pod \"e2e-worker-w1-d516c4-openshift-default-6bd584745-tjjpc\" (UID: \"a3128eb9-83aa-4ea0-abaf-c642b921870a\") " pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-tjjpc"
Oct 30 21:59:38.711694 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:38.711243    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a3128eb9-83aa-4ea0-abaf-c642b921870a-istiod-ca-cert\") pod \"e2e-worker-w1-d516c4-openshift-default-6bd584745-tjjpc\" (UID: \"a3128eb9-83aa-4ea0-abaf-c642b921870a\") " pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-tjjpc"
Oct 30 21:59:38.711694 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:38.711282    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3128eb9-83aa-4ea0-abaf-c642b921870a-credential-socket\") pod \"e2e-worker-w1-d516c4-openshift-default-6bd584745-tjjpc\" (UID: \"a3128eb9-83aa-4ea0-abaf-c642b921870a\") " pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-tjjpc"
Oct 30 21:59:38.711694 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:38.711314    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a3128eb9-83aa-4ea0-abaf-c642b921870a-istio-data\") pod \"e2e-worker-w1-d516c4-openshift-default-6bd584745-tjjpc\" (UID: \"a3128eb9-83aa-4ea0-abaf-c642b921870a\") " pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-tjjpc"
Oct 30 21:59:38.711694 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:38.711355    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a3128eb9-83aa-4ea0-abaf-c642b921870a-istio-podinfo\") pod \"e2e-worker-w1-d516c4-openshift-default-6bd584745-tjjpc\" (UID: \"a3128eb9-83aa-4ea0-abaf-c642b921870a\") " pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-tjjpc"
Oct 30 21:59:38.711694 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:38.711488    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3128eb9-83aa-4ea0-abaf-c642b921870a-workload-socket\") pod \"e2e-worker-w1-d516c4-openshift-default-6bd584745-tjjpc\" (UID: \"a3128eb9-83aa-4ea0-abaf-c642b921870a\") " pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-tjjpc"
Oct 30 21:59:38.712067 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:38.711732    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3128eb9-83aa-4ea0-abaf-c642b921870a-credential-socket\") pod \"e2e-worker-w1-d516c4-openshift-default-6bd584745-tjjpc\" (UID: \"a3128eb9-83aa-4ea0-abaf-c642b921870a\") " pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-tjjpc"
Oct 30 21:59:38.712067 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:38.711748    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a3128eb9-83aa-4ea0-abaf-c642b921870a-istio-data\") pod \"e2e-worker-w1-d516c4-openshift-default-6bd584745-tjjpc\" (UID: \"a3128eb9-83aa-4ea0-abaf-c642b921870a\") " pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-tjjpc"
Oct 30 21:59:38.713420 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:38.713396    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a3128eb9-83aa-4ea0-abaf-c642b921870a-istio-envoy\") pod \"e2e-worker-w1-d516c4-openshift-default-6bd584745-tjjpc\" (UID: \"a3128eb9-83aa-4ea0-abaf-c642b921870a\") " pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-tjjpc"
Oct 30 21:59:38.715044 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:38.715020    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a3128eb9-83aa-4ea0-abaf-c642b921870a-istio-podinfo\") pod \"e2e-worker-w1-d516c4-openshift-default-6bd584745-tjjpc\" (UID: \"a3128eb9-83aa-4ea0-abaf-c642b921870a\") " pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-tjjpc"
Oct 30 21:59:38.771886 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:38.771852    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vj7qj\" (UniqueName: \"kubernetes.io/projected/a3128eb9-83aa-4ea0-abaf-c642b921870a-kube-api-access-vj7qj\") pod \"e2e-worker-w1-d516c4-openshift-default-6bd584745-tjjpc\" (UID: \"a3128eb9-83aa-4ea0-abaf-c642b921870a\") " pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-tjjpc"
Oct 30 21:59:38.772090 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:38.771910    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a3128eb9-83aa-4ea0-abaf-c642b921870a-istio-token\") pod \"e2e-worker-w1-d516c4-openshift-default-6bd584745-tjjpc\" (UID: \"a3128eb9-83aa-4ea0-abaf-c642b921870a\") " pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-tjjpc"
Oct 30 21:59:38.858314 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:38.858272    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-2lzcz" event={"ID":"83504be1-81ee-4a97-93e3-a61ecb2f3b2d","Type":"ContainerStarted","Data":"16582c30f5a2afae5aac523619aa462ac28bc01d142bdc17c2215fde23c9d47e"}
Oct 30 21:59:38.858515 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:38.858343    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-2lzcz" event={"ID":"83504be1-81ee-4a97-93e3-a61ecb2f3b2d","Type":"ContainerStarted","Data":"54da5642fe8c061b3b5372de248e911939b297eccb115b71903371ed0069a407"}
Oct 30 21:59:38.860876 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:38.860841    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-vbr9j" event={"ID":"b9f9cb51-3fe0-43f3-9be3-95388ed892c8","Type":"ContainerStarted","Data":"29d33fff574cdf59f7fe25dbc0fc4a0badcfbe8a86d5cafaea44fea706d57817"}
Oct 30 21:59:38.861098 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:38.861023    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-vbr9j" event={"ID":"b9f9cb51-3fe0-43f3-9be3-95388ed892c8","Type":"ContainerStarted","Data":"e318c54d504541c8b537b6697d3fb2f924bfc6a0ec9ee0cc30f673f4451915b4"}
Oct 30 21:59:38.877435 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:38.877407    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-tjjpc"
Oct 30 21:59:38.916084 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:38.916048    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-chw2v"]
Oct 30 21:59:38.918135 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:59:38.918096    4473 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65b0ee0a_4761_40f3_bd4d_1b20dc75e233.slice/crio-3f4e911bbc235a413c55011a6d82fb2eab792aa36b807e7044dfad2bbe800203 WatchSource:0}: Error finding container 3f4e911bbc235a413c55011a6d82fb2eab792aa36b807e7044dfad2bbe800203: Status 404 returned error can't find the container with id 3f4e911bbc235a413c55011a6d82fb2eab792aa36b807e7044dfad2bbe800203
Oct 30 21:59:38.922482 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:38.922386    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-vbr9j"
Oct 30 21:59:38.924404 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:38.923786    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:59:38.924404 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:38.923886    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:59:38.924404 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:38.923956    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:59:38.988081 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:38.986347    4473 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-2lzcz" podStartSLOduration=1.98632506 podStartE2EDuration="1.98632506s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:59:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:59:38.98311068 +0000 UTC m=+1024.756479168" watchObservedRunningTime="2026-10-30 21:59:38.98632506 +0000 UTC m=+1024.759693548"
Oct 30 21:59:39.109367 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:39.109308    4473 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-vbr9j" podStartSLOduration=2.10929368 podStartE2EDuration="2.10929368s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:59:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:59:39.106362642 +0000 UTC m=+1024.879731129" watchObservedRunningTime="2026-10-30 21:59:39.10929368 +0000 UTC m=+1024.882662167"
Oct 30 21:59:39.151011 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:39.150971    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-2lzcz"
Oct 30 21:59:39.153544 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:39.153515    4473 patch_prober.go:29] interesting pod/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-2lzcz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body=
Oct 30 21:59:39.153719 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:39.153568    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-2lzcz" podUID="83504be1-81ee-4a97-93e3-a61ecb2f3b2d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused"
Oct 30 21:59:39.278914 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:39.278814    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-tjjpc"]
Oct 30 21:59:39.287130 ip-10-0-140-18 kubenswrapper[4473]: W1030 21:59:39.287098    4473 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3128eb9_83aa_4ea0_abaf_c642b921870a.slice/crio-f29c55d9f25c533a01d631750d7fb9b88ac38bc8147ea751f1808a69b4992b84 WatchSource:0}: Error finding container f29c55d9f25c533a01d631750d7fb9b88ac38bc8147ea751f1808a69b4992b84: Status 404 returned error can't find the container with id f29c55d9f25c533a01d631750d7fb9b88ac38bc8147ea751f1808a69b4992b84
Oct 30 21:59:39.289522 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:39.289478    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:59:39.289679 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:39.289567    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:59:39.289679 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:39.289613    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:59:39.900434 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:39.900394    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-tjjpc" event={"ID":"a3128eb9-83aa-4ea0-abaf-c642b921870a","Type":"ContainerStarted","Data":"0be752dc7462fe8fa74ef5ed5ad803a427ac7f621db2e23cb25d2cce2e34d9ad"}
Oct 30 21:59:39.900823 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:39.900464    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-tjjpc" event={"ID":"a3128eb9-83aa-4ea0-abaf-c642b921870a","Type":"ContainerStarted","Data":"f29c55d9f25c533a01d631750d7fb9b88ac38bc8147ea751f1808a69b4992b84"}
Oct 30 21:59:39.904033 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:39.903248    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-chw2v" event={"ID":"65b0ee0a-4761-40f3-bd4d-1b20dc75e233","Type":"ContainerStarted","Data":"4e96f42d773cab25ab394ce67e08b9e60080a9dab237b8cc30bc7e6d1a0cce25"}
Oct 30 21:59:39.904033 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:39.903310    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-chw2v" event={"ID":"65b0ee0a-4761-40f3-bd4d-1b20dc75e233","Type":"ContainerStarted","Data":"3f4e911bbc235a413c55011a6d82fb2eab792aa36b807e7044dfad2bbe800203"}
Oct 30 21:59:39.923308 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:39.923277    4473 patch_prober.go:29] interesting pod/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-vbr9j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body=
Oct 30 21:59:39.923455 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:39.923333    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-vbr9j" podUID="b9f9cb51-3fe0-43f3-9be3-95388ed892c8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused"
Oct 30 21:59:40.024857 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:40.024788    4473 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-tjjpc" podStartSLOduration=2.024767033 podStartE2EDuration="2.024767033s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:59:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:59:40.023484218 +0000 UTC m=+1025.796852732" watchObservedRunningTime="2026-10-30 21:59:40.024767033 +0000 UTC m=+1025.798135521"
Oct 30 21:59:40.165006 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:40.163959    4473 patch_prober.go:29] interesting pod/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-2lzcz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body=
Oct 30 21:59:40.165006 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:40.164026    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-2lzcz" podUID="83504be1-81ee-4a97-93e3-a61ecb2f3b2d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused"
Oct 30 21:59:40.167361 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:40.167298    4473 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-chw2v" podStartSLOduration=2.167277306 podStartE2EDuration="2.167277306s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:59:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:59:40.149322641 +0000 UTC m=+1025.922691130" watchObservedRunningTime="2026-10-30 21:59:40.167277306 +0000 UTC m=+1025.940645794"
Oct 30 21:59:40.520235 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:40.520128    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-chw2v"
Oct 30 21:59:40.520605 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:40.520580    4473 patch_prober.go:29] interesting pod/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-chw2v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body=
Oct 30 21:59:40.520798 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:40.520763    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-chw2v" podUID="65b0ee0a-4761-40f3-bd4d-1b20dc75e233" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused"
Oct 30 21:59:40.881720 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:40.881681    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-tjjpc"
Oct 30 21:59:40.882094 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:40.882071    4473 patch_prober.go:29] interesting pod/e2e-worker-w1-d516c4-openshift-default-6bd584745-tjjpc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body=
Oct 30 21:59:40.882277 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:40.882252    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-tjjpc" podUID="a3128eb9-83aa-4ea0-abaf-c642b921870a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused"
Oct 30 21:59:40.935734 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:40.935694    4473 patch_prober.go:29] interesting pod/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-vbr9j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body=
Oct 30 21:59:40.988946 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:40.981033    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-vbr9j" podUID="b9f9cb51-3fe0-43f3-9be3-95388ed892c8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused"
Oct 30 21:59:41.157008 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:41.155426    4473 patch_prober.go:29] interesting pod/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-2lzcz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body=
Oct 30 21:59:41.157008 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:41.155511    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-2lzcz" podUID="83504be1-81ee-4a97-93e3-a61ecb2f3b2d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused"
Oct 30 21:59:41.517440 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:41.517342    4473 patch_prober.go:29] interesting pod/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-chw2v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body=
Oct 30 21:59:41.517689 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:41.517660    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-chw2v" podUID="65b0ee0a-4761-40f3-bd4d-1b20dc75e233" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused"
Oct 30 21:59:41.879435 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:41.879394    4473 patch_prober.go:29] interesting pod/e2e-worker-w1-d516c4-openshift-default-6bd584745-tjjpc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body=
Oct 30 21:59:41.879745 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:41.879711    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-tjjpc" podUID="a3128eb9-83aa-4ea0-abaf-c642b921870a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused"
Oct 30 21:59:41.924949 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:41.922557    4473 patch_prober.go:29] interesting pod/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-vbr9j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body=
Oct 30 21:59:41.924949 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:41.922630    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-vbr9j" podUID="b9f9cb51-3fe0-43f3-9be3-95388ed892c8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused"
Oct 30 21:59:42.156287 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:42.156200    4473 patch_prober.go:29] interesting pod/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-2lzcz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body=
Oct 30 21:59:42.156840 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:42.156811    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-2lzcz" podUID="83504be1-81ee-4a97-93e3-a61ecb2f3b2d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused"
Oct 30 21:59:42.518493 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:42.518395    4473 patch_prober.go:29] interesting pod/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-chw2v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body=
Oct 30 21:59:42.519184 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:42.519147    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-chw2v" podUID="65b0ee0a-4761-40f3-bd4d-1b20dc75e233" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused"
Oct 30 21:59:42.883532 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:42.883496    4473 patch_prober.go:29] interesting pod/e2e-worker-w1-d516c4-openshift-default-6bd584745-tjjpc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body=
Oct 30 21:59:42.901728 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:42.883808    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-tjjpc" podUID="a3128eb9-83aa-4ea0-abaf-c642b921870a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused"
Oct 30 21:59:42.923527 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:42.923386    4473 patch_prober.go:29] interesting pod/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-vbr9j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body=
Oct 30 21:59:42.923527 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:42.923451    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-vbr9j" podUID="b9f9cb51-3fe0-43f3-9be3-95388ed892c8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused"
Oct 30 21:59:43.167477 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:43.167377    4473 patch_prober.go:29] interesting pod/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-2lzcz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body=
Oct 30 21:59:43.168084 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:43.168049    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-2lzcz" podUID="83504be1-81ee-4a97-93e3-a61ecb2f3b2d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused"
Oct 30 21:59:43.516707 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:43.516614    4473 patch_prober.go:29] interesting pod/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-chw2v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body=
Oct 30 21:59:43.516984 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:43.516954    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-chw2v" podUID="65b0ee0a-4761-40f3-bd4d-1b20dc75e233" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused"
Oct 30 21:59:43.892948 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:43.883250    4473 patch_prober.go:29] interesting pod/e2e-worker-w1-d516c4-openshift-default-6bd584745-tjjpc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body=
Oct 30 21:59:43.892948 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:43.883330    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-tjjpc" podUID="a3128eb9-83aa-4ea0-abaf-c642b921870a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused"
Oct 30 21:59:43.926310 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:43.926272    4473 patch_prober.go:29] interesting pod/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-vbr9j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body=
Oct 30 21:59:43.926585 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:43.926555    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-vbr9j" podUID="b9f9cb51-3fe0-43f3-9be3-95388ed892c8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused"
Oct 30 21:59:44.157596 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:44.157491    4473 patch_prober.go:29] interesting pod/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-2lzcz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body=
Oct 30 21:59:44.157868 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:44.157823    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-2lzcz" podUID="83504be1-81ee-4a97-93e3-a61ecb2f3b2d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused"
Oct 30 21:59:44.524078 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:44.519327    4473 patch_prober.go:29] interesting pod/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-chw2v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body=
Oct 30 21:59:44.524078 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:44.519387    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-chw2v" podUID="65b0ee0a-4761-40f3-bd4d-1b20dc75e233" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused"
Oct 30 21:59:44.901954 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:44.893144    4473 patch_prober.go:29] interesting pod/e2e-worker-w1-d516c4-openshift-default-6bd584745-tjjpc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body=
Oct 30 21:59:44.901954 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:44.893208    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-tjjpc" podUID="a3128eb9-83aa-4ea0-abaf-c642b921870a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused"
Oct 30 21:59:44.939361 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:44.939328    4473 patch_prober.go:29] interesting pod/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-vbr9j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body=
Oct 30 21:59:44.939647 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:44.939619    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-vbr9j" podUID="b9f9cb51-3fe0-43f3-9be3-95388ed892c8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused"
Oct 30 21:59:45.154438 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:45.154331    4473 patch_prober.go:29] interesting pod/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-2lzcz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body=
Oct 30 21:59:45.154625 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:45.154418    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-2lzcz" podUID="83504be1-81ee-4a97-93e3-a61ecb2f3b2d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused"
Oct 30 21:59:45.516423 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:45.516329    4473 patch_prober.go:29] interesting pod/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-chw2v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body=
Oct 30 21:59:45.516682 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:45.516650    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-chw2v" podUID="65b0ee0a-4761-40f3-bd4d-1b20dc75e233" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused"
Oct 30 21:59:45.900996 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:45.895426    4473 patch_prober.go:29] interesting pod/e2e-worker-w1-d516c4-openshift-default-6bd584745-tjjpc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body=
Oct 30 21:59:45.900996 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:45.895488    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-tjjpc" podUID="a3128eb9-83aa-4ea0-abaf-c642b921870a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused"
Oct 30 21:59:45.922298 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:45.922252    4473 patch_prober.go:29] interesting pod/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-vbr9j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body=
Oct 30 21:59:45.922490 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:45.922327    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-vbr9j" podUID="b9f9cb51-3fe0-43f3-9be3-95388ed892c8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused"
Oct 30 21:59:46.158461 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:46.158369    4473 patch_prober.go:29] interesting pod/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-2lzcz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body=
Oct 30 21:59:46.158742 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:46.158714    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-2lzcz" podUID="83504be1-81ee-4a97-93e3-a61ecb2f3b2d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused"
Oct 30 21:59:46.524904 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:46.523106    4473 patch_prober.go:29] interesting pod/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-chw2v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body=
Oct 30 21:59:46.524904 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:46.523168    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-chw2v" podUID="65b0ee0a-4761-40f3-bd4d-1b20dc75e233" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused"
Oct 30 21:59:46.879172 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:46.879000    4473 patch_prober.go:29] interesting pod/e2e-worker-w1-d516c4-openshift-default-6bd584745-tjjpc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body=
Oct 30 21:59:46.879172 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:46.879062    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-tjjpc" podUID="a3128eb9-83aa-4ea0-abaf-c642b921870a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused"
Oct 30 21:59:46.932027 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:46.931992    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-vbr9j"
Oct 30 21:59:46.934118 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:46.934086    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-vbr9j"
Oct 30 21:59:46.935314 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:46.935282    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-vbr9j"
Oct 30 21:59:47.157627 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:47.157538    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-2lzcz"
Oct 30 21:59:47.158022 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:47.157993    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-2lzcz"
Oct 30 21:59:47.158764 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:47.158741    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-2lzcz"
Oct 30 21:59:47.528371 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:47.528266    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-chw2v"
Oct 30 21:59:47.529332 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:47.529306    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-chw2v"
Oct 30 21:59:47.533327 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:47.533299    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-chw2v"
Oct 30 21:59:47.888985 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:47.888955    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-tjjpc"
Oct 30 21:59:47.890138 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:47.890106    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-tjjpc"
Oct 30 21:59:47.891405 ip-10-0-140-18 kubenswrapper[4473]: I1030 21:59:47.891384    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-tjjpc"
Oct 30 21:59:50.325283 ip-10-0-140-18 kubenswrapper[4473]: E1030 21:59:50.325247    4473 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fncvg" containerName="extractor"
Oct 30 22:00:00.203399 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:00:00.203361    4473 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-0c99b1-29889960-5rzls"]
Oct 30 22:00:00.207798 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:00:00.207771    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-0c99b1-29889960-5rzls"
Oct 30 22:00:00.222835 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:00:00.222806    4473 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-0f42cb-29889960-jnbj8"]
Oct 30 22:00:00.226870 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:00:00.226851    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-0f42cb-29889960-jnbj8"
Oct 30 22:00:00.279640 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:00:00.279605    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gdhc6\" (UniqueName: \"kubernetes.io/projected/75e13e43-c481-4933-b993-d7d7fcfa689e-kube-api-access-gdhc6\") pod \"maas-api-key-cleanup-e2e-worker-w3-0f42cb-29889960-jnbj8\" (UID: \"75e13e43-c481-4933-b993-d7d7fcfa689e\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-0f42cb-29889960-jnbj8"
Oct 30 22:00:00.280003 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:00:00.279973    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/75e13e43-c481-4933-b993-d7d7fcfa689e-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w3-0f42cb-29889960-jnbj8\" (UID: \"75e13e43-c481-4933-b993-d7d7fcfa689e\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-0f42cb-29889960-jnbj8"
Oct 30 22:00:00.280211 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:00:00.280183    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8qwg9\" (UniqueName: \"kubernetes.io/projected/10ea24b3-187c-4383-b6d2-8ae88ae60575-kube-api-access-8qwg9\") pod \"maas-api-key-cleanup-e2e-worker-w2-0c99b1-29889960-5rzls\" (UID: \"10ea24b3-187c-4383-b6d2-8ae88ae60575\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-0c99b1-29889960-5rzls"
Oct 30 22:00:00.280346 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:00:00.280329    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/10ea24b3-187c-4383-b6d2-8ae88ae60575-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w2-0c99b1-29889960-5rzls\" (UID: \"10ea24b3-187c-4383-b6d2-8ae88ae60575\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-0c99b1-29889960-5rzls"
Oct 30 22:00:00.319307 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:00:00.319275    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-0c99b1-29889960-5rzls"]
Oct 30 22:00:00.331843 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:00:00.331807    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-0f42cb-29889960-jnbj8"]
Oct 30 22:00:00.366160 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:00:00.366123    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-d786g\""
Oct 30 22:00:00.381953 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:00:00.381896    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8qwg9\" (UniqueName: \"kubernetes.io/projected/10ea24b3-187c-4383-b6d2-8ae88ae60575-kube-api-access-8qwg9\") pod \"maas-api-key-cleanup-e2e-worker-w2-0c99b1-29889960-5rzls\" (UID: \"10ea24b3-187c-4383-b6d2-8ae88ae60575\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-0c99b1-29889960-5rzls"
Oct 30 22:00:00.382150 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:00:00.382036    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/10ea24b3-187c-4383-b6d2-8ae88ae60575-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w2-0c99b1-29889960-5rzls\" (UID: \"10ea24b3-187c-4383-b6d2-8ae88ae60575\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-0c99b1-29889960-5rzls"
Oct 30 22:00:00.382150 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:00:00.382101    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gdhc6\" (UniqueName: \"kubernetes.io/projected/75e13e43-c481-4933-b993-d7d7fcfa689e-kube-api-access-gdhc6\") pod \"maas-api-key-cleanup-e2e-worker-w3-0f42cb-29889960-jnbj8\" (UID: \"75e13e43-c481-4933-b993-d7d7fcfa689e\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-0f42cb-29889960-jnbj8"
Oct 30 22:00:00.382150 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:00:00.382126    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/75e13e43-c481-4933-b993-d7d7fcfa689e-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w3-0f42cb-29889960-jnbj8\" (UID: \"75e13e43-c481-4933-b993-d7d7fcfa689e\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-0f42cb-29889960-jnbj8"
Oct 30 22:00:00.382983 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:00:00.382960    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/75e13e43-c481-4933-b993-d7d7fcfa689e-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w3-0f42cb-29889960-jnbj8\" (UID: \"75e13e43-c481-4933-b993-d7d7fcfa689e\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-0f42cb-29889960-jnbj8"
Oct 30 22:00:00.383048 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:00:00.382981    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/10ea24b3-187c-4383-b6d2-8ae88ae60575-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w2-0c99b1-29889960-5rzls\" (UID: \"10ea24b3-187c-4383-b6d2-8ae88ae60575\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-0c99b1-29889960-5rzls"
Oct 30 22:00:00.440380 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:00:00.440344    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8qwg9\" (UniqueName: \"kubernetes.io/projected/10ea24b3-187c-4383-b6d2-8ae88ae60575-kube-api-access-8qwg9\") pod \"maas-api-key-cleanup-e2e-worker-w2-0c99b1-29889960-5rzls\" (UID: \"10ea24b3-187c-4383-b6d2-8ae88ae60575\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-0c99b1-29889960-5rzls"
Oct 30 22:00:00.440537 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:00:00.440512    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gdhc6\" (UniqueName: \"kubernetes.io/projected/75e13e43-c481-4933-b993-d7d7fcfa689e-kube-api-access-gdhc6\") pod \"maas-api-key-cleanup-e2e-worker-w3-0f42cb-29889960-jnbj8\" (UID: \"75e13e43-c481-4933-b993-d7d7fcfa689e\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-0f42cb-29889960-jnbj8"
Oct 30 22:00:00.523814 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:00:00.523729    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-0c99b1-29889960-5rzls"
Oct 30 22:00:00.540030 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:00:00.540004    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-0f42cb-29889960-jnbj8"
Oct 30 22:00:00.910639 ip-10-0-140-18 kubenswrapper[4473]: W1030 22:00:00.910605    4473 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10ea24b3_187c_4383_b6d2_8ae88ae60575.slice/crio-d0a7bc9613f6f6c1606eb4d51e84dae01f60291ad5fbbf8976af05eedcd22c4a WatchSource:0}: Error finding container d0a7bc9613f6f6c1606eb4d51e84dae01f60291ad5fbbf8976af05eedcd22c4a: Status 404 returned error can't find the container with id d0a7bc9613f6f6c1606eb4d51e84dae01f60291ad5fbbf8976af05eedcd22c4a
Oct 30 22:00:00.910792 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:00:00.910771    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-0c99b1-29889960-5rzls"]
Oct 30 22:00:00.912718 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:00:00.912699    4473 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 22:00:00.960818 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:00:00.960792    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-0f42cb-29889960-jnbj8"]
Oct 30 22:00:00.961475 ip-10-0-140-18 kubenswrapper[4473]: W1030 22:00:00.961449    4473 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod75e13e43_c481_4933_b993_d7d7fcfa689e.slice/crio-448aa4a2cd5608bf37b1bf9e478854f9afd379509320c2dc097b3f88a12caa98 WatchSource:0}: Error finding container 448aa4a2cd5608bf37b1bf9e478854f9afd379509320c2dc097b3f88a12caa98: Status 404 returned error can't find the container with id 448aa4a2cd5608bf37b1bf9e478854f9afd379509320c2dc097b3f88a12caa98
Oct 30 22:00:01.529265 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:00:01.529219    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-0f42cb-29889960-jnbj8" event={"ID":"75e13e43-c481-4933-b993-d7d7fcfa689e","Type":"ContainerStarted","Data":"448aa4a2cd5608bf37b1bf9e478854f9afd379509320c2dc097b3f88a12caa98"}
Oct 30 22:00:01.530757 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:00:01.530725    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-0c99b1-29889960-5rzls" event={"ID":"10ea24b3-187c-4383-b6d2-8ae88ae60575","Type":"ContainerStarted","Data":"d0a7bc9613f6f6c1606eb4d51e84dae01f60291ad5fbbf8976af05eedcd22c4a"}
Oct 30 22:00:05.559669 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:00:05.559637    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-0f42cb-29889960-jnbj8" event={"ID":"75e13e43-c481-4933-b993-d7d7fcfa689e","Type":"ContainerStarted","Data":"dd3248cee7d55069666e80e7e80a10b4fc7d8b1ce8fe22266a65d54c32096f03"}
Oct 30 22:00:05.675593 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:00:05.675529    4473 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-0f42cb-29889960-jnbj8" podStartSLOduration=1.486187121 podStartE2EDuration="5.67551213s" totalImagesPullingTime="4.189325009s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:00:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:00:05.675175384 +0000 UTC m=+1051.448543871" watchObservedRunningTime="2026-10-30 22:00:05.67551213 +0000 UTC m=+1051.448880618"
Oct 30 22:00:06.576020 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:00:06.574052    4473 generic.go:356] "Generic (PLEG): container finished" podID="75e13e43-c481-4933-b993-d7d7fcfa689e" containerID="dd3248cee7d55069666e80e7e80a10b4fc7d8b1ce8fe22266a65d54c32096f03" exitCode=60
Oct 30 22:00:06.576020 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:00:06.574256    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-0f42cb-29889960-jnbj8" event={"ID":"75e13e43-c481-4933-b993-d7d7fcfa689e","Type":"ContainerDied","Data":"dd3248cee7d55069666e80e7e80a10b4fc7d8b1ce8fe22266a65d54c32096f03"}
Oct 30 22:00:06.576020 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:00:06.574774    4473 scope.go:122] "RemoveContainer" containerID="dd3248cee7d55069666e80e7e80a10b4fc7d8b1ce8fe22266a65d54c32096f03"
Oct 30 22:00:06.579806 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:00:06.579687    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-0c99b1-29889960-5rzls" event={"ID":"10ea24b3-187c-4383-b6d2-8ae88ae60575","Type":"ContainerStarted","Data":"e982f722a02098050b2645986aaea43ff01d481ac9a87cf9bc31d22f3a20b1d9"}
Oct 30 22:00:06.580322 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:00:06.580302    4473 scope.go:122] "RemoveContainer" containerID="e982f722a02098050b2645986aaea43ff01d481ac9a87cf9bc31d22f3a20b1d9"
Oct 30 22:00:07.593710 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:00:07.593605    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-0f42cb-29889960-jnbj8" event={"ID":"75e13e43-c481-4933-b993-d7d7fcfa689e","Type":"ContainerStarted","Data":"a808b3778d88aad98da9464f141f7122eeb5c3ea39c8daab7e100b9f49cbb6ad"}
Oct 30 22:00:07.597618 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:00:07.597591    4473 generic.go:356] "Generic (PLEG): container finished" podID="10ea24b3-187c-4383-b6d2-8ae88ae60575" containerID="e982f722a02098050b2645986aaea43ff01d481ac9a87cf9bc31d22f3a20b1d9" exitCode=60
Oct 30 22:00:07.597752 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:00:07.597676    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-0c99b1-29889960-5rzls" event={"ID":"10ea24b3-187c-4383-b6d2-8ae88ae60575","Type":"ContainerDied","Data":"e982f722a02098050b2645986aaea43ff01d481ac9a87cf9bc31d22f3a20b1d9"}
Oct 30 22:00:07.597752 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:00:07.597714    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-0c99b1-29889960-5rzls" event={"ID":"10ea24b3-187c-4383-b6d2-8ae88ae60575","Type":"ContainerStarted","Data":"27019faa258bbbbd25c992f93eb3b7ae180f2952d607bf1c8d49328dd1200041"}
Oct 30 22:00:08.605459 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:00:08.605425    4473 generic.go:356] "Generic (PLEG): container finished" podID="10ea24b3-187c-4383-b6d2-8ae88ae60575" containerID="27019faa258bbbbd25c992f93eb3b7ae180f2952d607bf1c8d49328dd1200041" exitCode=60
Oct 30 22:00:08.605953 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:00:08.605519    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-0c99b1-29889960-5rzls" event={"ID":"10ea24b3-187c-4383-b6d2-8ae88ae60575","Type":"ContainerDied","Data":"27019faa258bbbbd25c992f93eb3b7ae180f2952d607bf1c8d49328dd1200041"}
Oct 30 22:00:08.605953 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:00:08.605595    4473 scope.go:122] "RemoveContainer" containerID="e982f722a02098050b2645986aaea43ff01d481ac9a87cf9bc31d22f3a20b1d9"
Oct 30 22:00:08.605953 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:00:08.605777    4473 scope.go:122] "RemoveContainer" containerID="27019faa258bbbbd25c992f93eb3b7ae180f2952d607bf1c8d49328dd1200041"
Oct 30 22:00:08.606170 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:00:08.606118    4473 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w2-0c99b1-29889960-5rzls_odh-ai-gateway-infra(10ea24b3-187c-4383-b6d2-8ae88ae60575)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-0c99b1-29889960-5rzls" podUID="10ea24b3-187c-4383-b6d2-8ae88ae60575"
Oct 30 22:00:08.607677 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:00:08.607655    4473 generic.go:356] "Generic (PLEG): container finished" podID="75e13e43-c481-4933-b993-d7d7fcfa689e" containerID="a808b3778d88aad98da9464f141f7122eeb5c3ea39c8daab7e100b9f49cbb6ad" exitCode=60
Oct 30 22:00:08.607791 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:00:08.607733    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-0f42cb-29889960-jnbj8" event={"ID":"75e13e43-c481-4933-b993-d7d7fcfa689e","Type":"ContainerDied","Data":"a808b3778d88aad98da9464f141f7122eeb5c3ea39c8daab7e100b9f49cbb6ad"}
Oct 30 22:00:08.608004 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:00:08.607990    4473 scope.go:122] "RemoveContainer" containerID="a808b3778d88aad98da9464f141f7122eeb5c3ea39c8daab7e100b9f49cbb6ad"
Oct 30 22:00:08.608261 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:00:08.608246    4473 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w3-0f42cb-29889960-jnbj8_odh-ai-gateway-infra(75e13e43-c481-4933-b993-d7d7fcfa689e)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-0f42cb-29889960-jnbj8" podUID="75e13e43-c481-4933-b993-d7d7fcfa689e"
Oct 30 22:00:08.618129 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:00:08.618105    4473 scope.go:122] "RemoveContainer" containerID="dd3248cee7d55069666e80e7e80a10b4fc7d8b1ce8fe22266a65d54c32096f03"
Oct 30 22:00:09.616288 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:00:09.616262    4473 scope.go:122] "RemoveContainer" containerID="27019faa258bbbbd25c992f93eb3b7ae180f2952d607bf1c8d49328dd1200041"
Oct 30 22:00:09.616836 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:00:09.616609    4473 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w2-0c99b1-29889960-5rzls_odh-ai-gateway-infra(10ea24b3-187c-4383-b6d2-8ae88ae60575)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-0c99b1-29889960-5rzls" podUID="10ea24b3-187c-4383-b6d2-8ae88ae60575"
Oct 30 22:00:09.618023 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:00:09.618004    4473 scope.go:122] "RemoveContainer" containerID="a808b3778d88aad98da9464f141f7122eeb5c3ea39c8daab7e100b9f49cbb6ad"
Oct 30 22:00:09.618296 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:00:09.618274    4473 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w3-0f42cb-29889960-jnbj8_odh-ai-gateway-infra(75e13e43-c481-4933-b993-d7d7fcfa689e)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-0f42cb-29889960-jnbj8" podUID="75e13e43-c481-4933-b993-d7d7fcfa689e"
Oct 30 22:00:12.555128 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:00:12.555087    4473 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-dde1e8/distinct-w0-dde1e8-kserve-75b68b4c5d-hts2k"]
Oct 30 22:00:12.560739 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:00:12.560720    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-dde1e8/distinct-w0-dde1e8-kserve-75b68b4c5d-hts2k"
Oct 30 22:00:12.616174 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:00:12.616138    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-dde1e8/distinct-w0-dde1e8-kserve-75b68b4c5d-hts2k"]
Oct 30 22:00:12.663626 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:00:12.663596    4473 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-dde1e8\"/\"openshift-service-ca.crt\""
Oct 30 22:00:12.663626 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:00:12.663594    4473 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-dde1e8\"/\"kube-root-ca.crt\""
Oct 30 22:00:12.666884 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:00:12.666865    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-dde1e8\"/\"distinct-w0-dde1e8-kserve-self-signed-certs\""
Oct 30 22:00:12.667060 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:00:12.667044    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-dde1e8\"/\"default-dockercfg-zthlq\""
Oct 30 22:00:12.669794 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:00:12.669771    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9d72e7c9-657e-4278-8ed9-53adb75bd530-tmp-dir\") pod \"distinct-w0-dde1e8-kserve-75b68b4c5d-hts2k\" (UID: \"9d72e7c9-657e-4278-8ed9-53adb75bd530\") " pod="e2e-models-e2e-worker-w0-dde1e8/distinct-w0-dde1e8-kserve-75b68b4c5d-hts2k"
Oct 30 22:00:12.669896 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:00:12.669882    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9d72e7c9-657e-4278-8ed9-53adb75bd530-home\") pod \"distinct-w0-dde1e8-kserve-75b68b4c5d-hts2k\" (UID: \"9d72e7c9-657e-4278-8ed9-53adb75bd530\") " pod="e2e-models-e2e-worker-w0-dde1e8/distinct-w0-dde1e8-kserve-75b68b4c5d-hts2k"
Oct 30 22:00:12.669993 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:00:12.669981    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9d72e7c9-657e-4278-8ed9-53adb75bd530-dshm\") pod \"distinct-w0-dde1e8-kserve-75b68b4c5d-hts2k\" (UID: \"9d72e7c9-657e-4278-8ed9-53adb75bd530\") " pod="e2e-models-e2e-worker-w0-dde1e8/distinct-w0-dde1e8-kserve-75b68b4c5d-hts2k"
Oct 30 22:00:12.670037 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:00:12.670004    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9d72e7c9-657e-4278-8ed9-53adb75bd530-tls-certs\") pod \"distinct-w0-dde1e8-kserve-75b68b4c5d-hts2k\" (UID: \"9d72e7c9-657e-4278-8ed9-53adb75bd530\") " pod="e2e-models-e2e-worker-w0-dde1e8/distinct-w0-dde1e8-kserve-75b68b4c5d-hts2k"
Oct 30 22:00:12.670037 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:00:12.670029    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9d72e7c9-657e-4278-8ed9-53adb75bd530-model-cache\") pod \"distinct-w0-dde1e8-kserve-75b68b4c5d-hts2k\" (UID: \"9d72e7c9-657e-4278-8ed9-53adb75bd530\") " pod="e2e-models-e2e-worker-w0-dde1e8/distinct-w0-dde1e8-kserve-75b68b4c5d-hts2k"
Oct 30 22:00:12.670209 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:00:12.670191    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jvvdr\" (UniqueName: \"kubernetes.io/projected/9d72e7c9-657e-4278-8ed9-53adb75bd530-kube-api-access-jvvdr\") pod \"distinct-w0-dde1e8-kserve-75b68b4c5d-hts2k\" (UID: \"9d72e7c9-657e-4278-8ed9-53adb75bd530\") " pod="e2e-models-e2e-worker-w0-dde1e8/distinct-w0-dde1e8-kserve-75b68b4c5d-hts2k"
Oct 30 22:00:12.773837 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:00:12.773796    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9d72e7c9-657e-4278-8ed9-53adb75bd530-dshm\") pod \"distinct-w0-dde1e8-kserve-75b68b4c5d-hts2k\" (UID: \"9d72e7c9-657e-4278-8ed9-53adb75bd530\") " pod="e2e-models-e2e-worker-w0-dde1e8/distinct-w0-dde1e8-kserve-75b68b4c5d-hts2k"
Oct 30 22:00:12.774164 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:00:12.774140    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9d72e7c9-657e-4278-8ed9-53adb75bd530-tls-certs\") pod \"distinct-w0-dde1e8-kserve-75b68b4c5d-hts2k\" (UID: \"9d72e7c9-657e-4278-8ed9-53adb75bd530\") " pod="e2e-models-e2e-worker-w0-dde1e8/distinct-w0-dde1e8-kserve-75b68b4c5d-hts2k"
Oct 30 22:00:12.774307 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:00:12.774290    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9d72e7c9-657e-4278-8ed9-53adb75bd530-model-cache\") pod \"distinct-w0-dde1e8-kserve-75b68b4c5d-hts2k\" (UID: \"9d72e7c9-657e-4278-8ed9-53adb75bd530\") " pod="e2e-models-e2e-worker-w0-dde1e8/distinct-w0-dde1e8-kserve-75b68b4c5d-hts2k"
Oct 30 22:00:12.774606 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:00:12.774580    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jvvdr\" (UniqueName: \"kubernetes.io/projected/9d72e7c9-657e-4278-8ed9-53adb75bd530-kube-api-access-jvvdr\") pod \"distinct-w0-dde1e8-kserve-75b68b4c5d-hts2k\" (UID: \"9d72e7c9-657e-4278-8ed9-53adb75bd530\") " pod="e2e-models-e2e-worker-w0-dde1e8/distinct-w0-dde1e8-kserve-75b68b4c5d-hts2k"
Oct 30 22:00:12.774880 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:00:12.774862    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9d72e7c9-657e-4278-8ed9-53adb75bd530-tmp-dir\") pod \"distinct-w0-dde1e8-kserve-75b68b4c5d-hts2k\" (UID: \"9d72e7c9-657e-4278-8ed9-53adb75bd530\") " pod="e2e-models-e2e-worker-w0-dde1e8/distinct-w0-dde1e8-kserve-75b68b4c5d-hts2k"
Oct 30 22:00:12.775167 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:00:12.775149    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9d72e7c9-657e-4278-8ed9-53adb75bd530-home\") pod \"distinct-w0-dde1e8-kserve-75b68b4c5d-hts2k\" (UID: \"9d72e7c9-657e-4278-8ed9-53adb75bd530\") " pod="e2e-models-e2e-worker-w0-dde1e8/distinct-w0-dde1e8-kserve-75b68b4c5d-hts2k"
Oct 30 22:00:12.778936 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:00:12.775888    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9d72e7c9-657e-4278-8ed9-53adb75bd530-home\") pod \"distinct-w0-dde1e8-kserve-75b68b4c5d-hts2k\" (UID: \"9d72e7c9-657e-4278-8ed9-53adb75bd530\") " pod="e2e-models-e2e-worker-w0-dde1e8/distinct-w0-dde1e8-kserve-75b68b4c5d-hts2k"
Oct 30 22:00:12.779877 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:00:12.779458    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9d72e7c9-657e-4278-8ed9-53adb75bd530-model-cache\") pod \"distinct-w0-dde1e8-kserve-75b68b4c5d-hts2k\" (UID: \"9d72e7c9-657e-4278-8ed9-53adb75bd530\") " pod="e2e-models-e2e-worker-w0-dde1e8/distinct-w0-dde1e8-kserve-75b68b4c5d-hts2k"
Oct 30 22:00:12.780693 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:00:12.780652    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9d72e7c9-657e-4278-8ed9-53adb75bd530-tmp-dir\") pod \"distinct-w0-dde1e8-kserve-75b68b4c5d-hts2k\" (UID: \"9d72e7c9-657e-4278-8ed9-53adb75bd530\") " pod="e2e-models-e2e-worker-w0-dde1e8/distinct-w0-dde1e8-kserve-75b68b4c5d-hts2k"
Oct 30 22:00:12.785616 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:00:12.785582    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9d72e7c9-657e-4278-8ed9-53adb75bd530-tls-certs\") pod \"distinct-w0-dde1e8-kserve-75b68b4c5d-hts2k\" (UID: \"9d72e7c9-657e-4278-8ed9-53adb75bd530\") " pod="e2e-models-e2e-worker-w0-dde1e8/distinct-w0-dde1e8-kserve-75b68b4c5d-hts2k"
Oct 30 22:00:12.786881 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:00:12.786833    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9d72e7c9-657e-4278-8ed9-53adb75bd530-dshm\") pod \"distinct-w0-dde1e8-kserve-75b68b4c5d-hts2k\" (UID: \"9d72e7c9-657e-4278-8ed9-53adb75bd530\") " pod="e2e-models-e2e-worker-w0-dde1e8/distinct-w0-dde1e8-kserve-75b68b4c5d-hts2k"
Oct 30 22:00:12.838634 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:00:12.838557    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jvvdr\" (UniqueName: \"kubernetes.io/projected/9d72e7c9-657e-4278-8ed9-53adb75bd530-kube-api-access-jvvdr\") pod \"distinct-w0-dde1e8-kserve-75b68b4c5d-hts2k\" (UID: \"9d72e7c9-657e-4278-8ed9-53adb75bd530\") " pod="e2e-models-e2e-worker-w0-dde1e8/distinct-w0-dde1e8-kserve-75b68b4c5d-hts2k"
Oct 30 22:00:12.875115 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:00:12.875074    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-dde1e8/distinct-w0-dde1e8-kserve-75b68b4c5d-hts2k"
Oct 30 22:00:13.286770 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:00:13.286747    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-dde1e8/distinct-w0-dde1e8-kserve-75b68b4c5d-hts2k"]
Oct 30 22:00:13.287625 ip-10-0-140-18 kubenswrapper[4473]: W1030 22:00:13.287597    4473 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9d72e7c9_657e_4278_8ed9_53adb75bd530.slice/crio-b6d4387db14b77d3c4275d3439b480a5e9ecb1bdd6f2a5a99addb82db8c39cf5 WatchSource:0}: Error finding container b6d4387db14b77d3c4275d3439b480a5e9ecb1bdd6f2a5a99addb82db8c39cf5: Status 404 returned error can't find the container with id b6d4387db14b77d3c4275d3439b480a5e9ecb1bdd6f2a5a99addb82db8c39cf5
Oct 30 22:00:13.646039 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:00:13.645996    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-dde1e8/distinct-w0-dde1e8-kserve-75b68b4c5d-hts2k" event={"ID":"9d72e7c9-657e-4278-8ed9-53adb75bd530","Type":"ContainerStarted","Data":"b7be66dc5d185a6a713cc55f004ac81f8c2fa9bba4e8ab5e179e380b3591c0eb"}
Oct 30 22:00:13.646485 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:00:13.646049    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-dde1e8/distinct-w0-dde1e8-kserve-75b68b4c5d-hts2k" event={"ID":"9d72e7c9-657e-4278-8ed9-53adb75bd530","Type":"ContainerStarted","Data":"b6d4387db14b77d3c4275d3439b480a5e9ecb1bdd6f2a5a99addb82db8c39cf5"}
Oct 30 22:00:13.763445 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:00:13.763394    4473 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-dde1e8/distinct-w0-dde1e8-kserve-75b68b4c5d-hts2k" podStartSLOduration=1.763378906 podStartE2EDuration="1.763378906s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:00:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:00:13.762979397 +0000 UTC m=+1059.536347886" watchObservedRunningTime="2026-10-30 22:00:13.763378906 +0000 UTC m=+1059.536747393"
Oct 30 22:00:13.893121 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:00:13.892867    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-dde1e8/distinct-w0-dde1e8-kserve-75b68b4c5d-hts2k"
Oct 30 22:00:18.350142 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:00:18.350110    4473 scope.go:122] "RemoveContainer" containerID="a808b3778d88aad98da9464f141f7122eeb5c3ea39c8daab7e100b9f49cbb6ad"
Oct 30 22:00:18.682106 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:00:18.682075    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-0f42cb-29889960-jnbj8" event={"ID":"75e13e43-c481-4933-b993-d7d7fcfa689e","Type":"ContainerStarted","Data":"2736c77ed1f6042bd9d06873765b50549d338c6065285f4d8054b8364e2f71d1"}
Oct 30 22:00:19.691736 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:00:19.691703    4473 generic.go:356] "Generic (PLEG): container finished" podID="75e13e43-c481-4933-b993-d7d7fcfa689e" containerID="2736c77ed1f6042bd9d06873765b50549d338c6065285f4d8054b8364e2f71d1" exitCode=60
Oct 30 22:00:19.692292 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:00:19.691774    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-0f42cb-29889960-jnbj8" event={"ID":"75e13e43-c481-4933-b993-d7d7fcfa689e","Type":"ContainerDied","Data":"2736c77ed1f6042bd9d06873765b50549d338c6065285f4d8054b8364e2f71d1"}
Oct 30 22:00:19.692292 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:00:19.691876    4473 scope.go:122] "RemoveContainer" containerID="a808b3778d88aad98da9464f141f7122eeb5c3ea39c8daab7e100b9f49cbb6ad"
Oct 30 22:00:19.692413 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:00:19.692323    4473 scope.go:122] "RemoveContainer" containerID="2736c77ed1f6042bd9d06873765b50549d338c6065285f4d8054b8364e2f71d1"
Oct 30 22:00:19.692709 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:00:19.692681    4473 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w3-0f42cb-29889960-jnbj8_odh-ai-gateway-infra(75e13e43-c481-4933-b993-d7d7fcfa689e)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-0f42cb-29889960-jnbj8" podUID="75e13e43-c481-4933-b993-d7d7fcfa689e"
Oct 30 22:00:19.807067 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:00:19.807018    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-0f42cb-29889960-jnbj8"]
Oct 30 22:00:20.327694 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:00:20.327658    4473 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fncvg" containerName="extractor"
Oct 30 22:00:20.847475 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:00:20.847448    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-0f42cb-29889960-jnbj8"
Oct 30 22:00:20.993284 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:00:20.993193    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/75e13e43-c481-4933-b993-d7d7fcfa689e-kube-api-access-gdhc6\" (UniqueName: \"kubernetes.io/projected/75e13e43-c481-4933-b993-d7d7fcfa689e-kube-api-access-gdhc6\") pod \"75e13e43-c481-4933-b993-d7d7fcfa689e\" (UID: \"75e13e43-c481-4933-b993-d7d7fcfa689e\") "
Oct 30 22:00:20.993465 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:00:20.993417    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/75e13e43-c481-4933-b993-d7d7fcfa689e-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/75e13e43-c481-4933-b993-d7d7fcfa689e-maas-api-service-ca\") pod \"75e13e43-c481-4933-b993-d7d7fcfa689e\" (UID: \"75e13e43-c481-4933-b993-d7d7fcfa689e\") "
Oct 30 22:00:20.993871 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:00:20.993839    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/75e13e43-c481-4933-b993-d7d7fcfa689e-maas-api-service-ca" pod "75e13e43-c481-4933-b993-d7d7fcfa689e" (UID: "75e13e43-c481-4933-b993-d7d7fcfa689e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:00:20.995651 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:00:20.995619    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/75e13e43-c481-4933-b993-d7d7fcfa689e-kube-api-access-gdhc6" pod "75e13e43-c481-4933-b993-d7d7fcfa689e" (UID: "75e13e43-c481-4933-b993-d7d7fcfa689e"). InnerVolumeSpecName "kube-api-access-gdhc6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:00:21.095352 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:00:21.095314    4473 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gdhc6\" (UniqueName: \"kubernetes.io/projected/75e13e43-c481-4933-b993-d7d7fcfa689e-kube-api-access-gdhc6\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:00:21.095352 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:00:21.095349    4473 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/75e13e43-c481-4933-b993-d7d7fcfa689e-maas-api-service-ca\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:00:21.350031 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:00:21.349904    4473 scope.go:122] "RemoveContainer" containerID="27019faa258bbbbd25c992f93eb3b7ae180f2952d607bf1c8d49328dd1200041"
Oct 30 22:00:21.708149 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:00:21.708106    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-0c99b1-29889960-5rzls" event={"ID":"10ea24b3-187c-4383-b6d2-8ae88ae60575","Type":"ContainerStarted","Data":"9bbd631c5e07d79b26b351ebd4150fc31f787d8965124ac7e95af9a99e53d469"}
Oct 30 22:00:21.709440 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:00:21.709412    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-0f42cb-29889960-jnbj8" event={"ID":"75e13e43-c481-4933-b993-d7d7fcfa689e","Type":"ContainerDied","Data":"448aa4a2cd5608bf37b1bf9e478854f9afd379509320c2dc097b3f88a12caa98"}
Oct 30 22:00:21.709440 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:00:21.709430    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-0f42cb-29889960-jnbj8"
Oct 30 22:00:21.709632 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:00:21.709466    4473 scope.go:122] "RemoveContainer" containerID="2736c77ed1f6042bd9d06873765b50549d338c6065285f4d8054b8364e2f71d1"
Oct 30 22:00:21.825722 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:00:21.825660    4473 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-0c99b1-29889960-5rzls" podStartSLOduration=17.161059454 podStartE2EDuration="21.825644067s" totalImagesPullingTime="4.664584613s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:00:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:00:07.862725725 +0000 UTC m=+1053.636094214" watchObservedRunningTime="2026-10-30 22:00:21.825644067 +0000 UTC m=+1067.599012610"
Oct 30 22:00:21.938727 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:00:21.938641    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-0f42cb-29889960-jnbj8"]
Oct 30 22:00:21.943138 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:00:21.943097    4473 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-0f42cb-29889960-jnbj8"]
Oct 30 22:00:22.392974 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:00:22.391347    4473 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="75e13e43-c481-4933-b993-d7d7fcfa689e" path="/var/lib/kubelet/pods/75e13e43-c481-4933-b993-d7d7fcfa689e/volumes"
Oct 30 22:00:22.718695 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:00:22.718664    4473 generic.go:356] "Generic (PLEG): container finished" podID="10ea24b3-187c-4383-b6d2-8ae88ae60575" containerID="9bbd631c5e07d79b26b351ebd4150fc31f787d8965124ac7e95af9a99e53d469" exitCode=60
Oct 30 22:00:22.718908 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:00:22.718736    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-0c99b1-29889960-5rzls" event={"ID":"10ea24b3-187c-4383-b6d2-8ae88ae60575","Type":"ContainerDied","Data":"9bbd631c5e07d79b26b351ebd4150fc31f787d8965124ac7e95af9a99e53d469"}
Oct 30 22:00:22.718908 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:00:22.718795    4473 scope.go:122] "RemoveContainer" containerID="27019faa258bbbbd25c992f93eb3b7ae180f2952d607bf1c8d49328dd1200041"
Oct 30 22:00:22.719230 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:00:22.719199    4473 scope.go:122] "RemoveContainer" containerID="9bbd631c5e07d79b26b351ebd4150fc31f787d8965124ac7e95af9a99e53d469"
Oct 30 22:00:22.719582 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:00:22.719552    4473 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w2-0c99b1-29889960-5rzls_odh-ai-gateway-infra(10ea24b3-187c-4383-b6d2-8ae88ae60575)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-0c99b1-29889960-5rzls" podUID="10ea24b3-187c-4383-b6d2-8ae88ae60575"
Oct 30 22:00:22.835313 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:00:22.835265    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-0c99b1-29889960-5rzls"]
Oct 30 22:00:22.877826 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:00:22.875909    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-dde1e8/distinct-w0-dde1e8-kserve-75b68b4c5d-hts2k"
Oct 30 22:00:22.897678 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:00:22.897649    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-dde1e8/distinct-w0-dde1e8-kserve-75b68b4c5d-hts2k"
Oct 30 22:00:23.739672 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:00:23.739641    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-dde1e8/distinct-w0-dde1e8-kserve-75b68b4c5d-hts2k"
Oct 30 22:00:23.887994 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:00:23.887964    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-0c99b1-29889960-5rzls"
Oct 30 22:00:24.062350 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:00:24.062260    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/10ea24b3-187c-4383-b6d2-8ae88ae60575-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/10ea24b3-187c-4383-b6d2-8ae88ae60575-maas-api-service-ca\") pod \"10ea24b3-187c-4383-b6d2-8ae88ae60575\" (UID: \"10ea24b3-187c-4383-b6d2-8ae88ae60575\") "
Oct 30 22:00:24.062525 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:00:24.062440    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/10ea24b3-187c-4383-b6d2-8ae88ae60575-kube-api-access-8qwg9\" (UniqueName: \"kubernetes.io/projected/10ea24b3-187c-4383-b6d2-8ae88ae60575-kube-api-access-8qwg9\") pod \"10ea24b3-187c-4383-b6d2-8ae88ae60575\" (UID: \"10ea24b3-187c-4383-b6d2-8ae88ae60575\") "
Oct 30 22:00:24.062695 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:00:24.062669    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/10ea24b3-187c-4383-b6d2-8ae88ae60575-maas-api-service-ca" pod "10ea24b3-187c-4383-b6d2-8ae88ae60575" (UID: "10ea24b3-187c-4383-b6d2-8ae88ae60575"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:00:24.062934 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:00:24.062904    4473 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/10ea24b3-187c-4383-b6d2-8ae88ae60575-maas-api-service-ca\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:00:24.064696 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:00:24.064673    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/10ea24b3-187c-4383-b6d2-8ae88ae60575-kube-api-access-8qwg9" pod "10ea24b3-187c-4383-b6d2-8ae88ae60575" (UID: "10ea24b3-187c-4383-b6d2-8ae88ae60575"). InnerVolumeSpecName "kube-api-access-8qwg9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:00:24.163905 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:00:24.163871    4473 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8qwg9\" (UniqueName: \"kubernetes.io/projected/10ea24b3-187c-4383-b6d2-8ae88ae60575-kube-api-access-8qwg9\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:00:24.735819 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:00:24.735725    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-0c99b1-29889960-5rzls"
Oct 30 22:00:24.735819 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:00:24.735761    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-0c99b1-29889960-5rzls" event={"ID":"10ea24b3-187c-4383-b6d2-8ae88ae60575","Type":"ContainerDied","Data":"d0a7bc9613f6f6c1606eb4d51e84dae01f60291ad5fbbf8976af05eedcd22c4a"}
Oct 30 22:00:24.736075 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:00:24.735828    4473 scope.go:122] "RemoveContainer" containerID="9bbd631c5e07d79b26b351ebd4150fc31f787d8965124ac7e95af9a99e53d469"
Oct 30 22:00:24.905251 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:00:24.905221    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-0c99b1-29889960-5rzls"]
Oct 30 22:00:24.907780 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:00:24.907756    4473 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-0c99b1-29889960-5rzls"]
Oct 30 22:00:26.357634 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:00:26.357590    4473 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="10ea24b3-187c-4383-b6d2-8ae88ae60575" path="/var/lib/kubelet/pods/10ea24b3-187c-4383-b6d2-8ae88ae60575/volumes"
Oct 30 22:00:50.326779 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:00:50.326737    4473 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fncvg" containerName="extractor"
Oct 30 22:01:00.617987 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:00.617913    4473 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w3-0f42cb/distinct-w3-0f42cb-kserve-fb54fdd59-n6w4l"]
Oct 30 22:01:00.618522 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:00.618508    4473 cpu_manager.go:414] "containerMap: removing container" podUID="75e13e43-c481-4933-b993-d7d7fcfa689e" containerName="cleanup"
Oct 30 22:01:00.618571 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:00.618522    4473 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="75e13e43-c481-4933-b993-d7d7fcfa689e" containerName="cleanup"
Oct 30 22:01:00.618571 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:00.618538    4473 cpu_manager.go:414] "containerMap: removing container" podUID="75e13e43-c481-4933-b993-d7d7fcfa689e" containerName="cleanup"
Oct 30 22:01:00.618571 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:00.618543    4473 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="75e13e43-c481-4933-b993-d7d7fcfa689e" containerName="cleanup"
Oct 30 22:01:00.618571 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:00.618559    4473 cpu_manager.go:414] "containerMap: removing container" podUID="10ea24b3-187c-4383-b6d2-8ae88ae60575" containerName="cleanup"
Oct 30 22:01:00.618571 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:00.618565    4473 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="10ea24b3-187c-4383-b6d2-8ae88ae60575" containerName="cleanup"
Oct 30 22:01:00.618571 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:00.618570    4473 cpu_manager.go:414] "containerMap: removing container" podUID="10ea24b3-187c-4383-b6d2-8ae88ae60575" containerName="cleanup"
Oct 30 22:01:00.618571 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:00.618575    4473 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="10ea24b3-187c-4383-b6d2-8ae88ae60575" containerName="cleanup"
Oct 30 22:01:00.618876 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:00.618689    4473 memory_manager.go:365] "RemoveStaleState removing state" podUID="75e13e43-c481-4933-b993-d7d7fcfa689e" containerName="cleanup"
Oct 30 22:01:00.618876 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:00.618697    4473 memory_manager.go:365] "RemoveStaleState removing state" podUID="10ea24b3-187c-4383-b6d2-8ae88ae60575" containerName="cleanup"
Oct 30 22:01:00.618876 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:00.618705    4473 memory_manager.go:365] "RemoveStaleState removing state" podUID="75e13e43-c481-4933-b993-d7d7fcfa689e" containerName="cleanup"
Oct 30 22:01:00.618876 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:00.618711    4473 memory_manager.go:365] "RemoveStaleState removing state" podUID="10ea24b3-187c-4383-b6d2-8ae88ae60575" containerName="cleanup"
Oct 30 22:01:00.618876 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:00.618717    4473 memory_manager.go:365] "RemoveStaleState removing state" podUID="10ea24b3-187c-4383-b6d2-8ae88ae60575" containerName="cleanup"
Oct 30 22:01:00.622200 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:00.622181    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-0f42cb/distinct-w3-0f42cb-kserve-fb54fdd59-n6w4l"
Oct 30 22:01:00.726436 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:00.726398    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-0f42cb\"/\"default-dockercfg-8zswv\""
Oct 30 22:01:00.726662 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:00.726474    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-0f42cb\"/\"distinct-w3-0f42cb-kserve-self-signed-certs\""
Oct 30 22:01:00.726662 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:00.726474    4473 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w3-0f42cb\"/\"kube-root-ca.crt\""
Oct 30 22:01:00.726662 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:00.726561    4473 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w3-0f42cb\"/\"openshift-service-ca.crt\""
Oct 30 22:01:00.729355 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:00.729327    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-0f42cb/distinct-w3-0f42cb-kserve-fb54fdd59-n6w4l"]
Oct 30 22:01:00.797941 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:00.795814    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7qf9d\" (UniqueName: \"kubernetes.io/projected/ff3c2b25-08aa-4b70-bf57-9cf4a11d05ed-kube-api-access-7qf9d\") pod \"distinct-w3-0f42cb-kserve-fb54fdd59-n6w4l\" (UID: \"ff3c2b25-08aa-4b70-bf57-9cf4a11d05ed\") " pod="e2e-models-e2e-worker-w3-0f42cb/distinct-w3-0f42cb-kserve-fb54fdd59-n6w4l"
Oct 30 22:01:00.797941 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:00.795949    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ff3c2b25-08aa-4b70-bf57-9cf4a11d05ed-tls-certs\") pod \"distinct-w3-0f42cb-kserve-fb54fdd59-n6w4l\" (UID: \"ff3c2b25-08aa-4b70-bf57-9cf4a11d05ed\") " pod="e2e-models-e2e-worker-w3-0f42cb/distinct-w3-0f42cb-kserve-fb54fdd59-n6w4l"
Oct 30 22:01:00.797941 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:00.796101    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ff3c2b25-08aa-4b70-bf57-9cf4a11d05ed-home\") pod \"distinct-w3-0f42cb-kserve-fb54fdd59-n6w4l\" (UID: \"ff3c2b25-08aa-4b70-bf57-9cf4a11d05ed\") " pod="e2e-models-e2e-worker-w3-0f42cb/distinct-w3-0f42cb-kserve-fb54fdd59-n6w4l"
Oct 30 22:01:00.797941 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:00.796137    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ff3c2b25-08aa-4b70-bf57-9cf4a11d05ed-dshm\") pod \"distinct-w3-0f42cb-kserve-fb54fdd59-n6w4l\" (UID: \"ff3c2b25-08aa-4b70-bf57-9cf4a11d05ed\") " pod="e2e-models-e2e-worker-w3-0f42cb/distinct-w3-0f42cb-kserve-fb54fdd59-n6w4l"
Oct 30 22:01:00.797941 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:00.796183    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ff3c2b25-08aa-4b70-bf57-9cf4a11d05ed-model-cache\") pod \"distinct-w3-0f42cb-kserve-fb54fdd59-n6w4l\" (UID: \"ff3c2b25-08aa-4b70-bf57-9cf4a11d05ed\") " pod="e2e-models-e2e-worker-w3-0f42cb/distinct-w3-0f42cb-kserve-fb54fdd59-n6w4l"
Oct 30 22:01:00.797941 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:00.796324    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ff3c2b25-08aa-4b70-bf57-9cf4a11d05ed-tmp-dir\") pod \"distinct-w3-0f42cb-kserve-fb54fdd59-n6w4l\" (UID: \"ff3c2b25-08aa-4b70-bf57-9cf4a11d05ed\") " pod="e2e-models-e2e-worker-w3-0f42cb/distinct-w3-0f42cb-kserve-fb54fdd59-n6w4l"
Oct 30 22:01:00.897981 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:00.897830    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ff3c2b25-08aa-4b70-bf57-9cf4a11d05ed-tmp-dir\") pod \"distinct-w3-0f42cb-kserve-fb54fdd59-n6w4l\" (UID: \"ff3c2b25-08aa-4b70-bf57-9cf4a11d05ed\") " pod="e2e-models-e2e-worker-w3-0f42cb/distinct-w3-0f42cb-kserve-fb54fdd59-n6w4l"
Oct 30 22:01:00.897981 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:00.897968    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7qf9d\" (UniqueName: \"kubernetes.io/projected/ff3c2b25-08aa-4b70-bf57-9cf4a11d05ed-kube-api-access-7qf9d\") pod \"distinct-w3-0f42cb-kserve-fb54fdd59-n6w4l\" (UID: \"ff3c2b25-08aa-4b70-bf57-9cf4a11d05ed\") " pod="e2e-models-e2e-worker-w3-0f42cb/distinct-w3-0f42cb-kserve-fb54fdd59-n6w4l"
Oct 30 22:01:00.898229 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:00.898055    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ff3c2b25-08aa-4b70-bf57-9cf4a11d05ed-tls-certs\") pod \"distinct-w3-0f42cb-kserve-fb54fdd59-n6w4l\" (UID: \"ff3c2b25-08aa-4b70-bf57-9cf4a11d05ed\") " pod="e2e-models-e2e-worker-w3-0f42cb/distinct-w3-0f42cb-kserve-fb54fdd59-n6w4l"
Oct 30 22:01:00.898229 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:00.898124    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ff3c2b25-08aa-4b70-bf57-9cf4a11d05ed-home\") pod \"distinct-w3-0f42cb-kserve-fb54fdd59-n6w4l\" (UID: \"ff3c2b25-08aa-4b70-bf57-9cf4a11d05ed\") " pod="e2e-models-e2e-worker-w3-0f42cb/distinct-w3-0f42cb-kserve-fb54fdd59-n6w4l"
Oct 30 22:01:00.898229 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:00.898145    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ff3c2b25-08aa-4b70-bf57-9cf4a11d05ed-dshm\") pod \"distinct-w3-0f42cb-kserve-fb54fdd59-n6w4l\" (UID: \"ff3c2b25-08aa-4b70-bf57-9cf4a11d05ed\") " pod="e2e-models-e2e-worker-w3-0f42cb/distinct-w3-0f42cb-kserve-fb54fdd59-n6w4l"
Oct 30 22:01:00.898229 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:00.898184    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ff3c2b25-08aa-4b70-bf57-9cf4a11d05ed-model-cache\") pod \"distinct-w3-0f42cb-kserve-fb54fdd59-n6w4l\" (UID: \"ff3c2b25-08aa-4b70-bf57-9cf4a11d05ed\") " pod="e2e-models-e2e-worker-w3-0f42cb/distinct-w3-0f42cb-kserve-fb54fdd59-n6w4l"
Oct 30 22:01:00.898408 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:00.898313    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ff3c2b25-08aa-4b70-bf57-9cf4a11d05ed-tmp-dir\") pod \"distinct-w3-0f42cb-kserve-fb54fdd59-n6w4l\" (UID: \"ff3c2b25-08aa-4b70-bf57-9cf4a11d05ed\") " pod="e2e-models-e2e-worker-w3-0f42cb/distinct-w3-0f42cb-kserve-fb54fdd59-n6w4l"
Oct 30 22:01:00.898443 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:00.898421    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ff3c2b25-08aa-4b70-bf57-9cf4a11d05ed-home\") pod \"distinct-w3-0f42cb-kserve-fb54fdd59-n6w4l\" (UID: \"ff3c2b25-08aa-4b70-bf57-9cf4a11d05ed\") " pod="e2e-models-e2e-worker-w3-0f42cb/distinct-w3-0f42cb-kserve-fb54fdd59-n6w4l"
Oct 30 22:01:00.898579 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:00.898553    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ff3c2b25-08aa-4b70-bf57-9cf4a11d05ed-model-cache\") pod \"distinct-w3-0f42cb-kserve-fb54fdd59-n6w4l\" (UID: \"ff3c2b25-08aa-4b70-bf57-9cf4a11d05ed\") " pod="e2e-models-e2e-worker-w3-0f42cb/distinct-w3-0f42cb-kserve-fb54fdd59-n6w4l"
Oct 30 22:01:00.900974 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:00.900947    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ff3c2b25-08aa-4b70-bf57-9cf4a11d05ed-dshm\") pod \"distinct-w3-0f42cb-kserve-fb54fdd59-n6w4l\" (UID: \"ff3c2b25-08aa-4b70-bf57-9cf4a11d05ed\") " pod="e2e-models-e2e-worker-w3-0f42cb/distinct-w3-0f42cb-kserve-fb54fdd59-n6w4l"
Oct 30 22:01:00.901383 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:00.901360    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ff3c2b25-08aa-4b70-bf57-9cf4a11d05ed-tls-certs\") pod \"distinct-w3-0f42cb-kserve-fb54fdd59-n6w4l\" (UID: \"ff3c2b25-08aa-4b70-bf57-9cf4a11d05ed\") " pod="e2e-models-e2e-worker-w3-0f42cb/distinct-w3-0f42cb-kserve-fb54fdd59-n6w4l"
Oct 30 22:01:00.956080 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:00.956048    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7qf9d\" (UniqueName: \"kubernetes.io/projected/ff3c2b25-08aa-4b70-bf57-9cf4a11d05ed-kube-api-access-7qf9d\") pod \"distinct-w3-0f42cb-kserve-fb54fdd59-n6w4l\" (UID: \"ff3c2b25-08aa-4b70-bf57-9cf4a11d05ed\") " pod="e2e-models-e2e-worker-w3-0f42cb/distinct-w3-0f42cb-kserve-fb54fdd59-n6w4l"
Oct 30 22:01:01.235127 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:01.235039    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-0f42cb/distinct-w3-0f42cb-kserve-fb54fdd59-n6w4l"
Oct 30 22:01:01.611757 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:01.611726    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-0f42cb/distinct-w3-0f42cb-kserve-fb54fdd59-n6w4l"]
Oct 30 22:01:01.613093 ip-10-0-140-18 kubenswrapper[4473]: W1030 22:01:01.613066    4473 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podff3c2b25_08aa_4b70_bf57_9cf4a11d05ed.slice/crio-a2b0ce67416ac69520c8d12bfae61f0b69649d03128ed00a97bc1c0bba7c6b1c WatchSource:0}: Error finding container a2b0ce67416ac69520c8d12bfae61f0b69649d03128ed00a97bc1c0bba7c6b1c: Status 404 returned error can't find the container with id a2b0ce67416ac69520c8d12bfae61f0b69649d03128ed00a97bc1c0bba7c6b1c
Oct 30 22:01:02.064500 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:02.064457    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-0f42cb/distinct-w3-0f42cb-kserve-fb54fdd59-n6w4l" event={"ID":"ff3c2b25-08aa-4b70-bf57-9cf4a11d05ed","Type":"ContainerStarted","Data":"0135f7809e015e6c379e57672ed0828bcc1580e15a3b3fc617ec8462060533e4"}
Oct 30 22:01:02.064949 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:02.064510    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-0f42cb/distinct-w3-0f42cb-kserve-fb54fdd59-n6w4l" event={"ID":"ff3c2b25-08aa-4b70-bf57-9cf4a11d05ed","Type":"ContainerStarted","Data":"a2b0ce67416ac69520c8d12bfae61f0b69649d03128ed00a97bc1c0bba7c6b1c"}
Oct 30 22:01:02.186556 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:02.186496    4473 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w3-0f42cb/distinct-w3-0f42cb-kserve-fb54fdd59-n6w4l" podStartSLOduration=2.186479869 podStartE2EDuration="2.186479869s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:01:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:01:02.182594186 +0000 UTC m=+1107.955962673" watchObservedRunningTime="2026-10-30 22:01:02.186479869 +0000 UTC m=+1107.959848356"
Oct 30 22:01:02.236036 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:02.236001    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-0f42cb/distinct-w3-0f42cb-kserve-fb54fdd59-n6w4l"
Oct 30 22:01:08.910009 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:08.909972    4473 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-vncql"]
Oct 30 22:01:08.910630 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:08.910605    4473 cpu_manager.go:414] "containerMap: removing container" podUID="75e13e43-c481-4933-b993-d7d7fcfa689e" containerName="cleanup"
Oct 30 22:01:08.910630 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:08.910629    4473 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="75e13e43-c481-4933-b993-d7d7fcfa689e" containerName="cleanup"
Oct 30 22:01:08.910827 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:08.910675    4473 cpu_manager.go:414] "containerMap: removing container" podUID="10ea24b3-187c-4383-b6d2-8ae88ae60575" containerName="cleanup"
Oct 30 22:01:08.910827 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:08.910683    4473 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="10ea24b3-187c-4383-b6d2-8ae88ae60575" containerName="cleanup"
Oct 30 22:01:08.910827 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:08.910817    4473 memory_manager.go:365] "RemoveStaleState removing state" podUID="75e13e43-c481-4933-b993-d7d7fcfa689e" containerName="cleanup"
Oct 30 22:01:08.914347 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:08.914330    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-vncql"
Oct 30 22:01:09.061794 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:09.058182    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mpgfx\" (UniqueName: \"kubernetes.io/projected/06f032d9-4824-42fd-a92f-4cb00284ae32-kube-api-access-mpgfx\") pod \"e2e-worker-w6-29cbad-openshift-default-6d74d46b79-vncql\" (UID: \"06f032d9-4824-42fd-a92f-4cb00284ae32\") " pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-vncql"
Oct 30 22:01:09.061794 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:09.058294    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/06f032d9-4824-42fd-a92f-4cb00284ae32-istio-envoy\") pod \"e2e-worker-w6-29cbad-openshift-default-6d74d46b79-vncql\" (UID: \"06f032d9-4824-42fd-a92f-4cb00284ae32\") " pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-vncql"
Oct 30 22:01:09.061794 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:09.058354    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/06f032d9-4824-42fd-a92f-4cb00284ae32-istio-podinfo\") pod \"e2e-worker-w6-29cbad-openshift-default-6d74d46b79-vncql\" (UID: \"06f032d9-4824-42fd-a92f-4cb00284ae32\") " pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-vncql"
Oct 30 22:01:09.061794 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:09.058396    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/06f032d9-4824-42fd-a92f-4cb00284ae32-istiod-ca-cert\") pod \"e2e-worker-w6-29cbad-openshift-default-6d74d46b79-vncql\" (UID: \"06f032d9-4824-42fd-a92f-4cb00284ae32\") " pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-vncql"
Oct 30 22:01:09.061794 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:09.058444    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/06f032d9-4824-42fd-a92f-4cb00284ae32-credential-socket\") pod \"e2e-worker-w6-29cbad-openshift-default-6d74d46b79-vncql\" (UID: \"06f032d9-4824-42fd-a92f-4cb00284ae32\") " pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-vncql"
Oct 30 22:01:09.061794 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:09.058515    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/06f032d9-4824-42fd-a92f-4cb00284ae32-istio-token\") pod \"e2e-worker-w6-29cbad-openshift-default-6d74d46b79-vncql\" (UID: \"06f032d9-4824-42fd-a92f-4cb00284ae32\") " pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-vncql"
Oct 30 22:01:09.061794 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:09.058554    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/06f032d9-4824-42fd-a92f-4cb00284ae32-workload-certs\") pod \"e2e-worker-w6-29cbad-openshift-default-6d74d46b79-vncql\" (UID: \"06f032d9-4824-42fd-a92f-4cb00284ae32\") " pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-vncql"
Oct 30 22:01:09.061794 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:09.058575    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/06f032d9-4824-42fd-a92f-4cb00284ae32-istio-data\") pod \"e2e-worker-w6-29cbad-openshift-default-6d74d46b79-vncql\" (UID: \"06f032d9-4824-42fd-a92f-4cb00284ae32\") " pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-vncql"
Oct 30 22:01:09.061794 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:09.058653    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/06f032d9-4824-42fd-a92f-4cb00284ae32-workload-socket\") pod \"e2e-worker-w6-29cbad-openshift-default-6d74d46b79-vncql\" (UID: \"06f032d9-4824-42fd-a92f-4cb00284ae32\") " pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-vncql"
Oct 30 22:01:09.064827 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:09.064795    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-vncql"]
Oct 30 22:01:09.160178 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:09.160082    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mpgfx\" (UniqueName: \"kubernetes.io/projected/06f032d9-4824-42fd-a92f-4cb00284ae32-kube-api-access-mpgfx\") pod \"e2e-worker-w6-29cbad-openshift-default-6d74d46b79-vncql\" (UID: \"06f032d9-4824-42fd-a92f-4cb00284ae32\") " pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-vncql"
Oct 30 22:01:09.160178 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:09.160170    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/06f032d9-4824-42fd-a92f-4cb00284ae32-istio-envoy\") pod \"e2e-worker-w6-29cbad-openshift-default-6d74d46b79-vncql\" (UID: \"06f032d9-4824-42fd-a92f-4cb00284ae32\") " pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-vncql"
Oct 30 22:01:09.160423 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:09.160212    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/06f032d9-4824-42fd-a92f-4cb00284ae32-istio-podinfo\") pod \"e2e-worker-w6-29cbad-openshift-default-6d74d46b79-vncql\" (UID: \"06f032d9-4824-42fd-a92f-4cb00284ae32\") " pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-vncql"
Oct 30 22:01:09.160423 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:09.160239    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/06f032d9-4824-42fd-a92f-4cb00284ae32-istiod-ca-cert\") pod \"e2e-worker-w6-29cbad-openshift-default-6d74d46b79-vncql\" (UID: \"06f032d9-4824-42fd-a92f-4cb00284ae32\") " pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-vncql"
Oct 30 22:01:09.160423 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:09.160269    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/06f032d9-4824-42fd-a92f-4cb00284ae32-credential-socket\") pod \"e2e-worker-w6-29cbad-openshift-default-6d74d46b79-vncql\" (UID: \"06f032d9-4824-42fd-a92f-4cb00284ae32\") " pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-vncql"
Oct 30 22:01:09.160423 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:09.160334    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/06f032d9-4824-42fd-a92f-4cb00284ae32-istio-token\") pod \"e2e-worker-w6-29cbad-openshift-default-6d74d46b79-vncql\" (UID: \"06f032d9-4824-42fd-a92f-4cb00284ae32\") " pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-vncql"
Oct 30 22:01:09.160423 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:09.160373    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/06f032d9-4824-42fd-a92f-4cb00284ae32-workload-certs\") pod \"e2e-worker-w6-29cbad-openshift-default-6d74d46b79-vncql\" (UID: \"06f032d9-4824-42fd-a92f-4cb00284ae32\") " pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-vncql"
Oct 30 22:01:09.160423 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:09.160394    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/06f032d9-4824-42fd-a92f-4cb00284ae32-istio-data\") pod \"e2e-worker-w6-29cbad-openshift-default-6d74d46b79-vncql\" (UID: \"06f032d9-4824-42fd-a92f-4cb00284ae32\") " pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-vncql"
Oct 30 22:01:09.160691 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:09.160463    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/06f032d9-4824-42fd-a92f-4cb00284ae32-workload-socket\") pod \"e2e-worker-w6-29cbad-openshift-default-6d74d46b79-vncql\" (UID: \"06f032d9-4824-42fd-a92f-4cb00284ae32\") " pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-vncql"
Oct 30 22:01:09.160887 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:09.160850    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/06f032d9-4824-42fd-a92f-4cb00284ae32-istio-data\") pod \"e2e-worker-w6-29cbad-openshift-default-6d74d46b79-vncql\" (UID: \"06f032d9-4824-42fd-a92f-4cb00284ae32\") " pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-vncql"
Oct 30 22:01:09.160887 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:09.160885    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/06f032d9-4824-42fd-a92f-4cb00284ae32-workload-certs\") pod \"e2e-worker-w6-29cbad-openshift-default-6d74d46b79-vncql\" (UID: \"06f032d9-4824-42fd-a92f-4cb00284ae32\") " pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-vncql"
Oct 30 22:01:09.161261 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:09.160953    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/06f032d9-4824-42fd-a92f-4cb00284ae32-workload-socket\") pod \"e2e-worker-w6-29cbad-openshift-default-6d74d46b79-vncql\" (UID: \"06f032d9-4824-42fd-a92f-4cb00284ae32\") " pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-vncql"
Oct 30 22:01:09.161261 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:09.161082    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/06f032d9-4824-42fd-a92f-4cb00284ae32-istiod-ca-cert\") pod \"e2e-worker-w6-29cbad-openshift-default-6d74d46b79-vncql\" (UID: \"06f032d9-4824-42fd-a92f-4cb00284ae32\") " pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-vncql"
Oct 30 22:01:09.161261 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:09.161128    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/06f032d9-4824-42fd-a92f-4cb00284ae32-credential-socket\") pod \"e2e-worker-w6-29cbad-openshift-default-6d74d46b79-vncql\" (UID: \"06f032d9-4824-42fd-a92f-4cb00284ae32\") " pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-vncql"
Oct 30 22:01:09.162892 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:09.162866    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/06f032d9-4824-42fd-a92f-4cb00284ae32-istio-envoy\") pod \"e2e-worker-w6-29cbad-openshift-default-6d74d46b79-vncql\" (UID: \"06f032d9-4824-42fd-a92f-4cb00284ae32\") " pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-vncql"
Oct 30 22:01:09.163403 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:09.163382    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/06f032d9-4824-42fd-a92f-4cb00284ae32-istio-podinfo\") pod \"e2e-worker-w6-29cbad-openshift-default-6d74d46b79-vncql\" (UID: \"06f032d9-4824-42fd-a92f-4cb00284ae32\") " pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-vncql"
Oct 30 22:01:09.245976 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:09.243960    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mpgfx\" (UniqueName: \"kubernetes.io/projected/06f032d9-4824-42fd-a92f-4cb00284ae32-kube-api-access-mpgfx\") pod \"e2e-worker-w6-29cbad-openshift-default-6d74d46b79-vncql\" (UID: \"06f032d9-4824-42fd-a92f-4cb00284ae32\") " pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-vncql"
Oct 30 22:01:09.255604 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:09.255569    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/06f032d9-4824-42fd-a92f-4cb00284ae32-istio-token\") pod \"e2e-worker-w6-29cbad-openshift-default-6d74d46b79-vncql\" (UID: \"06f032d9-4824-42fd-a92f-4cb00284ae32\") " pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-vncql"
Oct 30 22:01:09.282948 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:09.277494    4473 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-xrf95"]
Oct 30 22:01:09.302790 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:09.302758    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-xrf95"
Oct 30 22:01:09.364604 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:09.364562    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ef9eca59-6c6f-4523-9046-4fd0611dacef-istiod-ca-cert\") pod \"e2e-worker-w1-d516c4-openshift-default-6bd584745-xrf95\" (UID: \"ef9eca59-6c6f-4523-9046-4fd0611dacef\") " pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-xrf95"
Oct 30 22:01:09.364795 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:09.364756    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ef9eca59-6c6f-4523-9046-4fd0611dacef-workload-certs\") pod \"e2e-worker-w1-d516c4-openshift-default-6bd584745-xrf95\" (UID: \"ef9eca59-6c6f-4523-9046-4fd0611dacef\") " pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-xrf95"
Oct 30 22:01:09.364795 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:09.364789    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ef9eca59-6c6f-4523-9046-4fd0611dacef-istio-envoy\") pod \"e2e-worker-w1-d516c4-openshift-default-6bd584745-xrf95\" (UID: \"ef9eca59-6c6f-4523-9046-4fd0611dacef\") " pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-xrf95"
Oct 30 22:01:09.365071 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:09.364989    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ef9eca59-6c6f-4523-9046-4fd0611dacef-credential-socket\") pod \"e2e-worker-w1-d516c4-openshift-default-6bd584745-xrf95\" (UID: \"ef9eca59-6c6f-4523-9046-4fd0611dacef\") " pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-xrf95"
Oct 30 22:01:09.365113 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:09.365097    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ef9eca59-6c6f-4523-9046-4fd0611dacef-istio-podinfo\") pod \"e2e-worker-w1-d516c4-openshift-default-6bd584745-xrf95\" (UID: \"ef9eca59-6c6f-4523-9046-4fd0611dacef\") " pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-xrf95"
Oct 30 22:01:09.365229 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:09.365211    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ef9eca59-6c6f-4523-9046-4fd0611dacef-workload-socket\") pod \"e2e-worker-w1-d516c4-openshift-default-6bd584745-xrf95\" (UID: \"ef9eca59-6c6f-4523-9046-4fd0611dacef\") " pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-xrf95"
Oct 30 22:01:09.365432 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:09.365416    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ef9eca59-6c6f-4523-9046-4fd0611dacef-istio-token\") pod \"e2e-worker-w1-d516c4-openshift-default-6bd584745-xrf95\" (UID: \"ef9eca59-6c6f-4523-9046-4fd0611dacef\") " pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-xrf95"
Oct 30 22:01:09.365593 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:09.365578    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ef9eca59-6c6f-4523-9046-4fd0611dacef-istio-data\") pod \"e2e-worker-w1-d516c4-openshift-default-6bd584745-xrf95\" (UID: \"ef9eca59-6c6f-4523-9046-4fd0611dacef\") " pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-xrf95"
Oct 30 22:01:09.365762 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:09.365734    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mm9mm\" (UniqueName: \"kubernetes.io/projected/ef9eca59-6c6f-4523-9046-4fd0611dacef-kube-api-access-mm9mm\") pod \"e2e-worker-w1-d516c4-openshift-default-6bd584745-xrf95\" (UID: \"ef9eca59-6c6f-4523-9046-4fd0611dacef\") " pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-xrf95"
Oct 30 22:01:09.366077 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:09.366060    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-xrf95"]
Oct 30 22:01:09.467334 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:09.467246    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ef9eca59-6c6f-4523-9046-4fd0611dacef-credential-socket\") pod \"e2e-worker-w1-d516c4-openshift-default-6bd584745-xrf95\" (UID: \"ef9eca59-6c6f-4523-9046-4fd0611dacef\") " pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-xrf95"
Oct 30 22:01:09.467509 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:09.467389    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ef9eca59-6c6f-4523-9046-4fd0611dacef-istio-podinfo\") pod \"e2e-worker-w1-d516c4-openshift-default-6bd584745-xrf95\" (UID: \"ef9eca59-6c6f-4523-9046-4fd0611dacef\") " pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-xrf95"
Oct 30 22:01:09.467509 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:09.467496    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ef9eca59-6c6f-4523-9046-4fd0611dacef-workload-socket\") pod \"e2e-worker-w1-d516c4-openshift-default-6bd584745-xrf95\" (UID: \"ef9eca59-6c6f-4523-9046-4fd0611dacef\") " pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-xrf95"
Oct 30 22:01:09.467637 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:09.467545    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ef9eca59-6c6f-4523-9046-4fd0611dacef-istio-token\") pod \"e2e-worker-w1-d516c4-openshift-default-6bd584745-xrf95\" (UID: \"ef9eca59-6c6f-4523-9046-4fd0611dacef\") " pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-xrf95"
Oct 30 22:01:09.467692 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:09.467632    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ef9eca59-6c6f-4523-9046-4fd0611dacef-istio-data\") pod \"e2e-worker-w1-d516c4-openshift-default-6bd584745-xrf95\" (UID: \"ef9eca59-6c6f-4523-9046-4fd0611dacef\") " pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-xrf95"
Oct 30 22:01:09.467692 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:09.467632    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ef9eca59-6c6f-4523-9046-4fd0611dacef-credential-socket\") pod \"e2e-worker-w1-d516c4-openshift-default-6bd584745-xrf95\" (UID: \"ef9eca59-6c6f-4523-9046-4fd0611dacef\") " pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-xrf95"
Oct 30 22:01:09.467794 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:09.467725    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mm9mm\" (UniqueName: \"kubernetes.io/projected/ef9eca59-6c6f-4523-9046-4fd0611dacef-kube-api-access-mm9mm\") pod \"e2e-worker-w1-d516c4-openshift-default-6bd584745-xrf95\" (UID: \"ef9eca59-6c6f-4523-9046-4fd0611dacef\") " pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-xrf95"
Oct 30 22:01:09.467794 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:09.467775    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ef9eca59-6c6f-4523-9046-4fd0611dacef-istiod-ca-cert\") pod \"e2e-worker-w1-d516c4-openshift-default-6bd584745-xrf95\" (UID: \"ef9eca59-6c6f-4523-9046-4fd0611dacef\") " pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-xrf95"
Oct 30 22:01:09.468003 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:09.467868    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ef9eca59-6c6f-4523-9046-4fd0611dacef-workload-certs\") pod \"e2e-worker-w1-d516c4-openshift-default-6bd584745-xrf95\" (UID: \"ef9eca59-6c6f-4523-9046-4fd0611dacef\") " pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-xrf95"
Oct 30 22:01:09.468003 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:09.467876    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ef9eca59-6c6f-4523-9046-4fd0611dacef-workload-socket\") pod \"e2e-worker-w1-d516c4-openshift-default-6bd584745-xrf95\" (UID: \"ef9eca59-6c6f-4523-9046-4fd0611dacef\") " pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-xrf95"
Oct 30 22:01:09.468003 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:09.467900    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ef9eca59-6c6f-4523-9046-4fd0611dacef-istio-envoy\") pod \"e2e-worker-w1-d516c4-openshift-default-6bd584745-xrf95\" (UID: \"ef9eca59-6c6f-4523-9046-4fd0611dacef\") " pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-xrf95"
Oct 30 22:01:09.468159 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:09.468093    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ef9eca59-6c6f-4523-9046-4fd0611dacef-istio-data\") pod \"e2e-worker-w1-d516c4-openshift-default-6bd584745-xrf95\" (UID: \"ef9eca59-6c6f-4523-9046-4fd0611dacef\") " pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-xrf95"
Oct 30 22:01:09.468316 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:09.468191    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ef9eca59-6c6f-4523-9046-4fd0611dacef-workload-certs\") pod \"e2e-worker-w1-d516c4-openshift-default-6bd584745-xrf95\" (UID: \"ef9eca59-6c6f-4523-9046-4fd0611dacef\") " pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-xrf95"
Oct 30 22:01:09.468723 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:09.468701    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ef9eca59-6c6f-4523-9046-4fd0611dacef-istiod-ca-cert\") pod \"e2e-worker-w1-d516c4-openshift-default-6bd584745-xrf95\" (UID: \"ef9eca59-6c6f-4523-9046-4fd0611dacef\") " pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-xrf95"
Oct 30 22:01:09.470233 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:09.470208    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ef9eca59-6c6f-4523-9046-4fd0611dacef-istio-podinfo\") pod \"e2e-worker-w1-d516c4-openshift-default-6bd584745-xrf95\" (UID: \"ef9eca59-6c6f-4523-9046-4fd0611dacef\") " pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-xrf95"
Oct 30 22:01:09.470353 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:09.470272    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ef9eca59-6c6f-4523-9046-4fd0611dacef-istio-envoy\") pod \"e2e-worker-w1-d516c4-openshift-default-6bd584745-xrf95\" (UID: \"ef9eca59-6c6f-4523-9046-4fd0611dacef\") " pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-xrf95"
Oct 30 22:01:09.529479 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:09.529443    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-vncql"
Oct 30 22:01:09.529702 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:09.529683    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ef9eca59-6c6f-4523-9046-4fd0611dacef-istio-token\") pod \"e2e-worker-w1-d516c4-openshift-default-6bd584745-xrf95\" (UID: \"ef9eca59-6c6f-4523-9046-4fd0611dacef\") " pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-xrf95"
Oct 30 22:01:09.529746 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:09.529704    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mm9mm\" (UniqueName: \"kubernetes.io/projected/ef9eca59-6c6f-4523-9046-4fd0611dacef-kube-api-access-mm9mm\") pod \"e2e-worker-w1-d516c4-openshift-default-6bd584745-xrf95\" (UID: \"ef9eca59-6c6f-4523-9046-4fd0611dacef\") " pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-xrf95"
Oct 30 22:01:09.669728 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:09.669689    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-xrf95"
Oct 30 22:01:09.963411 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:09.963382    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-vncql"]
Oct 30 22:01:09.964256 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:09.964209    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:01:09.964361 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:09.964339    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:01:09.964458 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:09.964442    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:01:10.074400 ip-10-0-140-18 kubenswrapper[4473]: W1030 22:01:10.074363    4473 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef9eca59_6c6f_4523_9046_4fd0611dacef.slice/crio-320cee48b509e6c096395caf16e940e026cadf470cca761e3aae132d084643bd WatchSource:0}: Error finding container 320cee48b509e6c096395caf16e940e026cadf470cca761e3aae132d084643bd: Status 404 returned error can't find the container with id 320cee48b509e6c096395caf16e940e026cadf470cca761e3aae132d084643bd
Oct 30 22:01:10.077415 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:10.077387    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-xrf95"]
Oct 30 22:01:10.077505 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:10.077470    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:01:10.077576 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:10.077557    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:01:10.077624 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:10.077609    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:01:10.115696 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:10.115660    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-vncql" event={"ID":"06f032d9-4824-42fd-a92f-4cb00284ae32","Type":"ContainerStarted","Data":"b411d21561d9608a68fa937208577fd21f399d91f5052522426b09803e7d6067"}
Oct 30 22:01:10.115870 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:10.115730    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-vncql" event={"ID":"06f032d9-4824-42fd-a92f-4cb00284ae32","Type":"ContainerStarted","Data":"9626db28aead5d161c54505b8eb500288a30cc40d25708e62f8ac4f952225012"}
Oct 30 22:01:10.117310 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:10.117283    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-xrf95" event={"ID":"ef9eca59-6c6f-4523-9046-4fd0611dacef","Type":"ContainerStarted","Data":"320cee48b509e6c096395caf16e940e026cadf470cca761e3aae132d084643bd"}
Oct 30 22:01:10.235904 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:10.235772    4473 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-vncql" podStartSLOduration=2.235754216 podStartE2EDuration="2.235754216s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:01:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:01:10.235013961 +0000 UTC m=+1116.008382451" watchObservedRunningTime="2026-10-30 22:01:10.235754216 +0000 UTC m=+1116.009122704"
Oct 30 22:01:10.532598 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:10.532498    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-vncql"
Oct 30 22:01:11.181100 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:11.181001    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-xrf95" event={"ID":"ef9eca59-6c6f-4523-9046-4fd0611dacef","Type":"ContainerStarted","Data":"16a4e0937aac7ea2443ad5fdf740a27f8021688faa118d0eaf746bd60f4229ca"}
Oct 30 22:01:11.239603 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:11.239564    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w3-0f42cb/distinct-w3-0f42cb-kserve-fb54fdd59-n6w4l"
Oct 30 22:01:11.408571 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:11.408546    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w3-0f42cb/distinct-w3-0f42cb-kserve-fb54fdd59-n6w4l"
Oct 30 22:01:11.544007 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:11.534020    4473 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-xrf95" podStartSLOduration=2.53400209 podStartE2EDuration="2.53400209s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:01:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:01:11.336362988 +0000 UTC m=+1117.109731476" watchObservedRunningTime="2026-10-30 22:01:11.53400209 +0000 UTC m=+1117.307370578"
Oct 30 22:01:11.544007 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:11.535138    4473 patch_prober.go:29] interesting pod/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-vncql container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused" start-of-body=
Oct 30 22:01:11.544007 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:11.535194    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-vncql" podUID="06f032d9-4824-42fd-a92f-4cb00284ae32" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused"
Oct 30 22:01:11.675008 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:11.674972    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-xrf95"
Oct 30 22:01:11.675416 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:11.675392    4473 patch_prober.go:29] interesting pod/e2e-worker-w1-d516c4-openshift-default-6bd584745-xrf95 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body=
Oct 30 22:01:11.675609 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:11.675582    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-xrf95" podUID="ef9eca59-6c6f-4523-9046-4fd0611dacef" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused"
Oct 30 22:01:12.215944 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:12.215894    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-0f42cb/distinct-w3-0f42cb-kserve-fb54fdd59-n6w4l"
Oct 30 22:01:12.550016 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:12.547514    4473 patch_prober.go:29] interesting pod/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-vncql container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused" start-of-body=
Oct 30 22:01:12.550016 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:12.547593    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-vncql" podUID="06f032d9-4824-42fd-a92f-4cb00284ae32" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused"
Oct 30 22:01:12.675548 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:12.675500    4473 patch_prober.go:29] interesting pod/e2e-worker-w1-d516c4-openshift-default-6bd584745-xrf95 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body=
Oct 30 22:01:12.675757 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:12.675598    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-xrf95" podUID="ef9eca59-6c6f-4523-9046-4fd0611dacef" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused"
Oct 30 22:01:13.540948 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:13.534397    4473 patch_prober.go:29] interesting pod/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-vncql container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused" start-of-body=
Oct 30 22:01:13.540948 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:13.534465    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-vncql" podUID="06f032d9-4824-42fd-a92f-4cb00284ae32" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused"
Oct 30 22:01:13.672948 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:13.671767    4473 patch_prober.go:29] interesting pod/e2e-worker-w1-d516c4-openshift-default-6bd584745-xrf95 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body=
Oct 30 22:01:13.672948 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:13.671833    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-xrf95" podUID="ef9eca59-6c6f-4523-9046-4fd0611dacef" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused"
Oct 30 22:01:14.533948 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:14.530456    4473 patch_prober.go:29] interesting pod/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-vncql container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused" start-of-body=
Oct 30 22:01:14.533948 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:14.530519    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-vncql" podUID="06f032d9-4824-42fd-a92f-4cb00284ae32" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused"
Oct 30 22:01:14.673433 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:14.673394    4473 patch_prober.go:29] interesting pod/e2e-worker-w1-d516c4-openshift-default-6bd584745-xrf95 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body=
Oct 30 22:01:14.674481 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:14.674049    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-xrf95" podUID="ef9eca59-6c6f-4523-9046-4fd0611dacef" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused"
Oct 30 22:01:15.538403 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:15.538361    4473 patch_prober.go:29] interesting pod/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-vncql container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused" start-of-body=
Oct 30 22:01:15.538675 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:15.538642    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-vncql" podUID="06f032d9-4824-42fd-a92f-4cb00284ae32" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused"
Oct 30 22:01:15.679647 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:15.679610    4473 patch_prober.go:29] interesting pod/e2e-worker-w1-d516c4-openshift-default-6bd584745-xrf95 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body=
Oct 30 22:01:15.680283 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:15.680246    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-xrf95" podUID="ef9eca59-6c6f-4523-9046-4fd0611dacef" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused"
Oct 30 22:01:16.530879 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:16.530832    4473 patch_prober.go:29] interesting pod/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-vncql container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused" start-of-body=
Oct 30 22:01:16.531116 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:16.530896    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-vncql" podUID="06f032d9-4824-42fd-a92f-4cb00284ae32" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused"
Oct 30 22:01:16.670635 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:16.670601    4473 patch_prober.go:29] interesting pod/e2e-worker-w1-d516c4-openshift-default-6bd584745-xrf95 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body=
Oct 30 22:01:16.670806 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:16.670662    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-xrf95" podUID="ef9eca59-6c6f-4523-9046-4fd0611dacef" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused"
Oct 30 22:01:17.534764 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:17.534728    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-vncql"
Oct 30 22:01:17.535182 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:17.535131    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-vncql"
Oct 30 22:01:17.535856 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:17.535833    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-vncql"
Oct 30 22:01:17.675760 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:17.675729    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-xrf95"
Oct 30 22:01:17.676062 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:17.676044    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-xrf95"
Oct 30 22:01:17.676693 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:17.676675    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-xrf95"
Oct 30 22:01:20.324943 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:01:20.324887    4473 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fncvg" containerName="extractor"
Oct 30 22:01:39.831019 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:39.830983    4473 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-wrxqs"]
Oct 30 22:01:39.835546 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:39.835523    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-wrxqs"
Oct 30 22:01:39.932364 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:39.932329    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-wrxqs"]
Oct 30 22:01:39.983503 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:39.983464    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/370d0319-cf46-40d6-8c55-e4fa23e33dd3-istio-token\") pod \"e2e-worker-w3-0f42cb-openshift-default-584f4c4865-wrxqs\" (UID: \"370d0319-cf46-40d6-8c55-e4fa23e33dd3\") " pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-wrxqs"
Oct 30 22:01:39.983690 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:39.983602    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/370d0319-cf46-40d6-8c55-e4fa23e33dd3-istio-podinfo\") pod \"e2e-worker-w3-0f42cb-openshift-default-584f4c4865-wrxqs\" (UID: \"370d0319-cf46-40d6-8c55-e4fa23e33dd3\") " pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-wrxqs"
Oct 30 22:01:39.983690 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:39.983629    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/370d0319-cf46-40d6-8c55-e4fa23e33dd3-istiod-ca-cert\") pod \"e2e-worker-w3-0f42cb-openshift-default-584f4c4865-wrxqs\" (UID: \"370d0319-cf46-40d6-8c55-e4fa23e33dd3\") " pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-wrxqs"
Oct 30 22:01:39.983690 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:39.983666    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/370d0319-cf46-40d6-8c55-e4fa23e33dd3-workload-certs\") pod \"e2e-worker-w3-0f42cb-openshift-default-584f4c4865-wrxqs\" (UID: \"370d0319-cf46-40d6-8c55-e4fa23e33dd3\") " pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-wrxqs"
Oct 30 22:01:39.983793 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:39.983699    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/370d0319-cf46-40d6-8c55-e4fa23e33dd3-istio-data\") pod \"e2e-worker-w3-0f42cb-openshift-default-584f4c4865-wrxqs\" (UID: \"370d0319-cf46-40d6-8c55-e4fa23e33dd3\") " pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-wrxqs"
Oct 30 22:01:39.983793 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:39.983765    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/370d0319-cf46-40d6-8c55-e4fa23e33dd3-workload-socket\") pod \"e2e-worker-w3-0f42cb-openshift-default-584f4c4865-wrxqs\" (UID: \"370d0319-cf46-40d6-8c55-e4fa23e33dd3\") " pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-wrxqs"
Oct 30 22:01:39.983858 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:39.983814    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/370d0319-cf46-40d6-8c55-e4fa23e33dd3-credential-socket\") pod \"e2e-worker-w3-0f42cb-openshift-default-584f4c4865-wrxqs\" (UID: \"370d0319-cf46-40d6-8c55-e4fa23e33dd3\") " pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-wrxqs"
Oct 30 22:01:39.983977 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:39.983954    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/370d0319-cf46-40d6-8c55-e4fa23e33dd3-istio-envoy\") pod \"e2e-worker-w3-0f42cb-openshift-default-584f4c4865-wrxqs\" (UID: \"370d0319-cf46-40d6-8c55-e4fa23e33dd3\") " pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-wrxqs"
Oct 30 22:01:39.984079 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:39.984063    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ttzs8\" (UniqueName: \"kubernetes.io/projected/370d0319-cf46-40d6-8c55-e4fa23e33dd3-kube-api-access-ttzs8\") pod \"e2e-worker-w3-0f42cb-openshift-default-584f4c4865-wrxqs\" (UID: \"370d0319-cf46-40d6-8c55-e4fa23e33dd3\") " pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-wrxqs"
Oct 30 22:01:40.085703 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:40.085615    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/370d0319-cf46-40d6-8c55-e4fa23e33dd3-istio-podinfo\") pod \"e2e-worker-w3-0f42cb-openshift-default-584f4c4865-wrxqs\" (UID: \"370d0319-cf46-40d6-8c55-e4fa23e33dd3\") " pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-wrxqs"
Oct 30 22:01:40.085703 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:40.085674    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/370d0319-cf46-40d6-8c55-e4fa23e33dd3-istiod-ca-cert\") pod \"e2e-worker-w3-0f42cb-openshift-default-584f4c4865-wrxqs\" (UID: \"370d0319-cf46-40d6-8c55-e4fa23e33dd3\") " pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-wrxqs"
Oct 30 22:01:40.085909 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:40.085716    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/370d0319-cf46-40d6-8c55-e4fa23e33dd3-workload-certs\") pod \"e2e-worker-w3-0f42cb-openshift-default-584f4c4865-wrxqs\" (UID: \"370d0319-cf46-40d6-8c55-e4fa23e33dd3\") " pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-wrxqs"
Oct 30 22:01:40.085909 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:40.085739    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/370d0319-cf46-40d6-8c55-e4fa23e33dd3-istio-data\") pod \"e2e-worker-w3-0f42cb-openshift-default-584f4c4865-wrxqs\" (UID: \"370d0319-cf46-40d6-8c55-e4fa23e33dd3\") " pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-wrxqs"
Oct 30 22:01:40.085909 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:40.085774    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/370d0319-cf46-40d6-8c55-e4fa23e33dd3-workload-socket\") pod \"e2e-worker-w3-0f42cb-openshift-default-584f4c4865-wrxqs\" (UID: \"370d0319-cf46-40d6-8c55-e4fa23e33dd3\") " pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-wrxqs"
Oct 30 22:01:40.085909 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:40.085848    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/370d0319-cf46-40d6-8c55-e4fa23e33dd3-credential-socket\") pod \"e2e-worker-w3-0f42cb-openshift-default-584f4c4865-wrxqs\" (UID: \"370d0319-cf46-40d6-8c55-e4fa23e33dd3\") " pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-wrxqs"
Oct 30 22:01:40.086142 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:40.085939    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/370d0319-cf46-40d6-8c55-e4fa23e33dd3-istio-envoy\") pod \"e2e-worker-w3-0f42cb-openshift-default-584f4c4865-wrxqs\" (UID: \"370d0319-cf46-40d6-8c55-e4fa23e33dd3\") " pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-wrxqs"
Oct 30 22:01:40.086318 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:40.086263    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/370d0319-cf46-40d6-8c55-e4fa23e33dd3-istio-data\") pod \"e2e-worker-w3-0f42cb-openshift-default-584f4c4865-wrxqs\" (UID: \"370d0319-cf46-40d6-8c55-e4fa23e33dd3\") " pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-wrxqs"
Oct 30 22:01:40.086457 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:40.086352    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ttzs8\" (UniqueName: \"kubernetes.io/projected/370d0319-cf46-40d6-8c55-e4fa23e33dd3-kube-api-access-ttzs8\") pod \"e2e-worker-w3-0f42cb-openshift-default-584f4c4865-wrxqs\" (UID: \"370d0319-cf46-40d6-8c55-e4fa23e33dd3\") " pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-wrxqs"
Oct 30 22:01:40.086457 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:40.086419    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/370d0319-cf46-40d6-8c55-e4fa23e33dd3-workload-socket\") pod \"e2e-worker-w3-0f42cb-openshift-default-584f4c4865-wrxqs\" (UID: \"370d0319-cf46-40d6-8c55-e4fa23e33dd3\") " pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-wrxqs"
Oct 30 22:01:40.086457 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:40.086357    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/370d0319-cf46-40d6-8c55-e4fa23e33dd3-workload-certs\") pod \"e2e-worker-w3-0f42cb-openshift-default-584f4c4865-wrxqs\" (UID: \"370d0319-cf46-40d6-8c55-e4fa23e33dd3\") " pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-wrxqs"
Oct 30 22:01:40.086457 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:40.086430    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/370d0319-cf46-40d6-8c55-e4fa23e33dd3-istio-token\") pod \"e2e-worker-w3-0f42cb-openshift-default-584f4c4865-wrxqs\" (UID: \"370d0319-cf46-40d6-8c55-e4fa23e33dd3\") " pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-wrxqs"
Oct 30 22:01:40.086457 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:40.086437    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/370d0319-cf46-40d6-8c55-e4fa23e33dd3-credential-socket\") pod \"e2e-worker-w3-0f42cb-openshift-default-584f4c4865-wrxqs\" (UID: \"370d0319-cf46-40d6-8c55-e4fa23e33dd3\") " pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-wrxqs"
Oct 30 22:01:40.086956 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:40.086636    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/370d0319-cf46-40d6-8c55-e4fa23e33dd3-istiod-ca-cert\") pod \"e2e-worker-w3-0f42cb-openshift-default-584f4c4865-wrxqs\" (UID: \"370d0319-cf46-40d6-8c55-e4fa23e33dd3\") " pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-wrxqs"
Oct 30 22:01:40.088298 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:40.088276    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/370d0319-cf46-40d6-8c55-e4fa23e33dd3-istio-envoy\") pod \"e2e-worker-w3-0f42cb-openshift-default-584f4c4865-wrxqs\" (UID: \"370d0319-cf46-40d6-8c55-e4fa23e33dd3\") " pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-wrxqs"
Oct 30 22:01:40.088421 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:40.088402    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/370d0319-cf46-40d6-8c55-e4fa23e33dd3-istio-podinfo\") pod \"e2e-worker-w3-0f42cb-openshift-default-584f4c4865-wrxqs\" (UID: \"370d0319-cf46-40d6-8c55-e4fa23e33dd3\") " pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-wrxqs"
Oct 30 22:01:40.145161 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:40.145127    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/370d0319-cf46-40d6-8c55-e4fa23e33dd3-istio-token\") pod \"e2e-worker-w3-0f42cb-openshift-default-584f4c4865-wrxqs\" (UID: \"370d0319-cf46-40d6-8c55-e4fa23e33dd3\") " pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-wrxqs"
Oct 30 22:01:40.145328 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:40.145196    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ttzs8\" (UniqueName: \"kubernetes.io/projected/370d0319-cf46-40d6-8c55-e4fa23e33dd3-kube-api-access-ttzs8\") pod \"e2e-worker-w3-0f42cb-openshift-default-584f4c4865-wrxqs\" (UID: \"370d0319-cf46-40d6-8c55-e4fa23e33dd3\") " pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-wrxqs"
Oct 30 22:01:40.151021 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:40.151000    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-wrxqs"
Oct 30 22:01:40.587378 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:40.587339    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-wrxqs"]
Oct 30 22:01:40.591742 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:40.591707    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:01:40.591823 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:40.591776    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:01:40.591823 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:40.591806    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:01:41.454754 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:41.454713    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-wrxqs" event={"ID":"370d0319-cf46-40d6-8c55-e4fa23e33dd3","Type":"ContainerStarted","Data":"9b945b7ecd947aae012a9611e1357fb1c653c55a4b5206e9ea6c92930624a0a2"}
Oct 30 22:01:41.455269 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:41.454778    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-wrxqs" event={"ID":"370d0319-cf46-40d6-8c55-e4fa23e33dd3","Type":"ContainerStarted","Data":"322c5185bd2e5730c85ce3c9ac57326674f9bab74410d3adf6d320133f945e91"}
Oct 30 22:01:41.631947 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:41.627682    4473 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-wrxqs" podStartSLOduration=2.627662356 podStartE2EDuration="2.627662356s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:01:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:01:41.624189971 +0000 UTC m=+1147.397558461" watchObservedRunningTime="2026-10-30 22:01:41.627662356 +0000 UTC m=+1147.401030845"
Oct 30 22:01:42.151956 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:42.151420    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-wrxqs"
Oct 30 22:01:42.153136 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:42.153107    4473 patch_prober.go:29] interesting pod/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-wrxqs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused" start-of-body=
Oct 30 22:01:42.153362 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:42.153334    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-wrxqs" podUID="370d0319-cf46-40d6-8c55-e4fa23e33dd3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused"
Oct 30 22:01:43.151793 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:43.151748    4473 patch_prober.go:29] interesting pod/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-wrxqs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused" start-of-body=
Oct 30 22:01:43.152368 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:43.151827    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-wrxqs" podUID="370d0319-cf46-40d6-8c55-e4fa23e33dd3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused"
Oct 30 22:01:44.151645 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:44.151609    4473 patch_prober.go:29] interesting pod/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-wrxqs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused" start-of-body=
Oct 30 22:01:44.151953 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:44.151665    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-wrxqs" podUID="370d0319-cf46-40d6-8c55-e4fa23e33dd3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused"
Oct 30 22:01:45.151541 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:45.151498    4473 patch_prober.go:29] interesting pod/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-wrxqs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused" start-of-body=
Oct 30 22:01:45.151770 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:45.151561    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-wrxqs" podUID="370d0319-cf46-40d6-8c55-e4fa23e33dd3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused"
Oct 30 22:01:46.151461 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:46.151423    4473 patch_prober.go:29] interesting pod/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-wrxqs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused" start-of-body=
Oct 30 22:01:46.152027 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:46.151482    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-wrxqs" podUID="370d0319-cf46-40d6-8c55-e4fa23e33dd3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused"
Oct 30 22:01:47.151953 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:47.151884    4473 patch_prober.go:29] interesting pod/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-wrxqs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused" start-of-body=
Oct 30 22:01:47.152372 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:47.151969    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-wrxqs" podUID="370d0319-cf46-40d6-8c55-e4fa23e33dd3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused"
Oct 30 22:01:48.156296 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:48.156267    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-wrxqs"
Oct 30 22:01:48.156691 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:48.156673    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-wrxqs"
Oct 30 22:01:48.157410 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:01:48.157394    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-wrxqs"
Oct 30 22:01:50.326217 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:01:50.326181    4473 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fncvg" containerName="extractor"
Oct 30 22:02:08.391518 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:02:08.391481    4473 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-jsdrm"]
Oct 30 22:02:08.395794 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:02:08.395772    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-jsdrm"
Oct 30 22:02:08.500735 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:02:08.500693    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-jsdrm"]
Oct 30 22:02:08.520692 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:02:08.520648    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f100a752-adf7-4593-b9cb-9a4b276473c4-istio-envoy\") pod \"e2e-worker-w2-0c99b1-openshift-default-967855977-jsdrm\" (UID: \"f100a752-adf7-4593-b9cb-9a4b276473c4\") " pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-jsdrm"
Oct 30 22:02:08.520903 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:02:08.520713    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f100a752-adf7-4593-b9cb-9a4b276473c4-workload-certs\") pod \"e2e-worker-w2-0c99b1-openshift-default-967855977-jsdrm\" (UID: \"f100a752-adf7-4593-b9cb-9a4b276473c4\") " pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-jsdrm"
Oct 30 22:02:08.520903 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:02:08.520870    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f100a752-adf7-4593-b9cb-9a4b276473c4-workload-socket\") pod \"e2e-worker-w2-0c99b1-openshift-default-967855977-jsdrm\" (UID: \"f100a752-adf7-4593-b9cb-9a4b276473c4\") " pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-jsdrm"
Oct 30 22:02:08.521119 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:02:08.520902    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f100a752-adf7-4593-b9cb-9a4b276473c4-istiod-ca-cert\") pod \"e2e-worker-w2-0c99b1-openshift-default-967855977-jsdrm\" (UID: \"f100a752-adf7-4593-b9cb-9a4b276473c4\") " pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-jsdrm"
Oct 30 22:02:08.521119 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:02:08.521059    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f100a752-adf7-4593-b9cb-9a4b276473c4-istio-data\") pod \"e2e-worker-w2-0c99b1-openshift-default-967855977-jsdrm\" (UID: \"f100a752-adf7-4593-b9cb-9a4b276473c4\") " pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-jsdrm"
Oct 30 22:02:08.521119 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:02:08.521093    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f100a752-adf7-4593-b9cb-9a4b276473c4-istio-podinfo\") pod \"e2e-worker-w2-0c99b1-openshift-default-967855977-jsdrm\" (UID: \"f100a752-adf7-4593-b9cb-9a4b276473c4\") " pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-jsdrm"
Oct 30 22:02:08.521119 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:02:08.521116    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f100a752-adf7-4593-b9cb-9a4b276473c4-istio-token\") pod \"e2e-worker-w2-0c99b1-openshift-default-967855977-jsdrm\" (UID: \"f100a752-adf7-4593-b9cb-9a4b276473c4\") " pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-jsdrm"
Oct 30 22:02:08.521334 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:02:08.521245    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zt5mt\" (UniqueName: \"kubernetes.io/projected/f100a752-adf7-4593-b9cb-9a4b276473c4-kube-api-access-zt5mt\") pod \"e2e-worker-w2-0c99b1-openshift-default-967855977-jsdrm\" (UID: \"f100a752-adf7-4593-b9cb-9a4b276473c4\") " pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-jsdrm"
Oct 30 22:02:08.521393 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:02:08.521334    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f100a752-adf7-4593-b9cb-9a4b276473c4-credential-socket\") pod \"e2e-worker-w2-0c99b1-openshift-default-967855977-jsdrm\" (UID: \"f100a752-adf7-4593-b9cb-9a4b276473c4\") " pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-jsdrm"
Oct 30 22:02:08.622423 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:02:08.622384    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f100a752-adf7-4593-b9cb-9a4b276473c4-credential-socket\") pod \"e2e-worker-w2-0c99b1-openshift-default-967855977-jsdrm\" (UID: \"f100a752-adf7-4593-b9cb-9a4b276473c4\") " pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-jsdrm"
Oct 30 22:02:08.622629 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:02:08.622477    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f100a752-adf7-4593-b9cb-9a4b276473c4-istio-envoy\") pod \"e2e-worker-w2-0c99b1-openshift-default-967855977-jsdrm\" (UID: \"f100a752-adf7-4593-b9cb-9a4b276473c4\") " pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-jsdrm"
Oct 30 22:02:08.622629 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:02:08.622507    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f100a752-adf7-4593-b9cb-9a4b276473c4-workload-certs\") pod \"e2e-worker-w2-0c99b1-openshift-default-967855977-jsdrm\" (UID: \"f100a752-adf7-4593-b9cb-9a4b276473c4\") " pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-jsdrm"
Oct 30 22:02:08.622629 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:02:08.622563    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f100a752-adf7-4593-b9cb-9a4b276473c4-workload-socket\") pod \"e2e-worker-w2-0c99b1-openshift-default-967855977-jsdrm\" (UID: \"f100a752-adf7-4593-b9cb-9a4b276473c4\") " pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-jsdrm"
Oct 30 22:02:08.622629 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:02:08.622589    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f100a752-adf7-4593-b9cb-9a4b276473c4-istiod-ca-cert\") pod \"e2e-worker-w2-0c99b1-openshift-default-967855977-jsdrm\" (UID: \"f100a752-adf7-4593-b9cb-9a4b276473c4\") " pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-jsdrm"
Oct 30 22:02:08.622852 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:02:08.622642    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f100a752-adf7-4593-b9cb-9a4b276473c4-istio-data\") pod \"e2e-worker-w2-0c99b1-openshift-default-967855977-jsdrm\" (UID: \"f100a752-adf7-4593-b9cb-9a4b276473c4\") " pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-jsdrm"
Oct 30 22:02:08.622852 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:02:08.622672    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f100a752-adf7-4593-b9cb-9a4b276473c4-istio-podinfo\") pod \"e2e-worker-w2-0c99b1-openshift-default-967855977-jsdrm\" (UID: \"f100a752-adf7-4593-b9cb-9a4b276473c4\") " pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-jsdrm"
Oct 30 22:02:08.622852 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:02:08.622697    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f100a752-adf7-4593-b9cb-9a4b276473c4-istio-token\") pod \"e2e-worker-w2-0c99b1-openshift-default-967855977-jsdrm\" (UID: \"f100a752-adf7-4593-b9cb-9a4b276473c4\") " pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-jsdrm"
Oct 30 22:02:08.622852 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:02:08.622729    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zt5mt\" (UniqueName: \"kubernetes.io/projected/f100a752-adf7-4593-b9cb-9a4b276473c4-kube-api-access-zt5mt\") pod \"e2e-worker-w2-0c99b1-openshift-default-967855977-jsdrm\" (UID: \"f100a752-adf7-4593-b9cb-9a4b276473c4\") " pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-jsdrm"
Oct 30 22:02:08.623112 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:02:08.622859    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f100a752-adf7-4593-b9cb-9a4b276473c4-credential-socket\") pod \"e2e-worker-w2-0c99b1-openshift-default-967855977-jsdrm\" (UID: \"f100a752-adf7-4593-b9cb-9a4b276473c4\") " pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-jsdrm"
Oct 30 22:02:08.623112 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:02:08.622990    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f100a752-adf7-4593-b9cb-9a4b276473c4-workload-socket\") pod \"e2e-worker-w2-0c99b1-openshift-default-967855977-jsdrm\" (UID: \"f100a752-adf7-4593-b9cb-9a4b276473c4\") " pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-jsdrm"
Oct 30 22:02:08.623220 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:02:08.623109    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f100a752-adf7-4593-b9cb-9a4b276473c4-istio-data\") pod \"e2e-worker-w2-0c99b1-openshift-default-967855977-jsdrm\" (UID: \"f100a752-adf7-4593-b9cb-9a4b276473c4\") " pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-jsdrm"
Oct 30 22:02:08.623220 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:02:08.623169    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f100a752-adf7-4593-b9cb-9a4b276473c4-workload-certs\") pod \"e2e-worker-w2-0c99b1-openshift-default-967855977-jsdrm\" (UID: \"f100a752-adf7-4593-b9cb-9a4b276473c4\") " pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-jsdrm"
Oct 30 22:02:08.623437 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:02:08.623409    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f100a752-adf7-4593-b9cb-9a4b276473c4-istiod-ca-cert\") pod \"e2e-worker-w2-0c99b1-openshift-default-967855977-jsdrm\" (UID: \"f100a752-adf7-4593-b9cb-9a4b276473c4\") " pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-jsdrm"
Oct 30 22:02:08.625083 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:02:08.625062    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f100a752-adf7-4593-b9cb-9a4b276473c4-istio-envoy\") pod \"e2e-worker-w2-0c99b1-openshift-default-967855977-jsdrm\" (UID: \"f100a752-adf7-4593-b9cb-9a4b276473c4\") " pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-jsdrm"
Oct 30 22:02:08.625358 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:02:08.625339    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f100a752-adf7-4593-b9cb-9a4b276473c4-istio-podinfo\") pod \"e2e-worker-w2-0c99b1-openshift-default-967855977-jsdrm\" (UID: \"f100a752-adf7-4593-b9cb-9a4b276473c4\") " pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-jsdrm"
Oct 30 22:02:08.681755 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:02:08.681673    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f100a752-adf7-4593-b9cb-9a4b276473c4-istio-token\") pod \"e2e-worker-w2-0c99b1-openshift-default-967855977-jsdrm\" (UID: \"f100a752-adf7-4593-b9cb-9a4b276473c4\") " pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-jsdrm"
Oct 30 22:02:08.681896 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:02:08.681752    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zt5mt\" (UniqueName: \"kubernetes.io/projected/f100a752-adf7-4593-b9cb-9a4b276473c4-kube-api-access-zt5mt\") pod \"e2e-worker-w2-0c99b1-openshift-default-967855977-jsdrm\" (UID: \"f100a752-adf7-4593-b9cb-9a4b276473c4\") " pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-jsdrm"
Oct 30 22:02:08.711391 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:02:08.711353    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-jsdrm"
Oct 30 22:02:09.148506 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:02:09.148479    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-jsdrm"]
Oct 30 22:02:09.153522 ip-10-0-140-18 kubenswrapper[4473]: W1030 22:02:09.153486    4473 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf100a752_adf7_4593_b9cb_9a4b276473c4.slice/crio-ccd00b8f71de8c9884d7a0130c8808b8cd6dcbbc80343b962b47693336cd5690 WatchSource:0}: Error finding container ccd00b8f71de8c9884d7a0130c8808b8cd6dcbbc80343b962b47693336cd5690: Status 404 returned error can't find the container with id ccd00b8f71de8c9884d7a0130c8808b8cd6dcbbc80343b962b47693336cd5690
Oct 30 22:02:09.155523 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:02:09.155488    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:02:09.155618 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:02:09.155563    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:02:09.155618 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:02:09.155592    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:02:09.685282 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:02:09.685242    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-jsdrm" event={"ID":"f100a752-adf7-4593-b9cb-9a4b276473c4","Type":"ContainerStarted","Data":"de4d576a5ae5e7ccba6f1ff4d4c50cb825c54004dc84421151be2927ad215739"}
Oct 30 22:02:09.685712 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:02:09.685303    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-jsdrm" event={"ID":"f100a752-adf7-4593-b9cb-9a4b276473c4","Type":"ContainerStarted","Data":"ccd00b8f71de8c9884d7a0130c8808b8cd6dcbbc80343b962b47693336cd5690"}
Oct 30 22:02:09.712224 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:02:09.712191    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-jsdrm"
Oct 30 22:02:09.811723 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:02:09.811657    4473 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-jsdrm" podStartSLOduration=1.811635076 podStartE2EDuration="1.811635076s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:02:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:02:09.808256214 +0000 UTC m=+1175.581624702" watchObservedRunningTime="2026-10-30 22:02:09.811635076 +0000 UTC m=+1175.585003565"
Oct 30 22:02:10.714118 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:02:10.714083    4473 patch_prober.go:29] interesting pod/e2e-worker-w2-0c99b1-openshift-default-967855977-jsdrm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" start-of-body=
Oct 30 22:02:10.714543 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:02:10.714136    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-jsdrm" podUID="f100a752-adf7-4593-b9cb-9a4b276473c4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused"
Oct 30 22:02:11.712644 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:02:11.712603    4473 patch_prober.go:29] interesting pod/e2e-worker-w2-0c99b1-openshift-default-967855977-jsdrm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" start-of-body=
Oct 30 22:02:11.712837 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:02:11.712696    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-jsdrm" podUID="f100a752-adf7-4593-b9cb-9a4b276473c4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused"
Oct 30 22:02:12.712474 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:02:12.712436    4473 patch_prober.go:29] interesting pod/e2e-worker-w2-0c99b1-openshift-default-967855977-jsdrm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" start-of-body=
Oct 30 22:02:12.712898 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:02:12.712507    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-jsdrm" podUID="f100a752-adf7-4593-b9cb-9a4b276473c4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused"
Oct 30 22:02:13.711949 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:02:13.711892    4473 patch_prober.go:29] interesting pod/e2e-worker-w2-0c99b1-openshift-default-967855977-jsdrm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" start-of-body=
Oct 30 22:02:13.712143 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:02:13.711985    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-jsdrm" podUID="f100a752-adf7-4593-b9cb-9a4b276473c4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused"
Oct 30 22:02:14.712313 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:02:14.712283    4473 patch_prober.go:29] interesting pod/e2e-worker-w2-0c99b1-openshift-default-967855977-jsdrm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" start-of-body=
Oct 30 22:02:14.712762 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:02:14.712337    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-jsdrm" podUID="f100a752-adf7-4593-b9cb-9a4b276473c4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused"
Oct 30 22:02:15.712462 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:02:15.712420    4473 patch_prober.go:29] interesting pod/e2e-worker-w2-0c99b1-openshift-default-967855977-jsdrm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" start-of-body=
Oct 30 22:02:15.712869 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:02:15.712479    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-jsdrm" podUID="f100a752-adf7-4593-b9cb-9a4b276473c4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused"
Oct 30 22:02:16.716537 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:02:16.716500    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-jsdrm"
Oct 30 22:02:16.717040 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:02:16.717023    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-jsdrm"
Oct 30 22:02:16.717909 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:02:16.717888    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-jsdrm"
Oct 30 22:02:20.338244 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:02:20.338202    4473 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fncvg" containerName="extractor"
Oct 30 22:02:23.868019 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:02:23.867984    4473 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-ch5zt"]
Oct 30 22:02:23.872219 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:02:23.872191    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-ch5zt"
Oct 30 22:02:23.945101 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:02:23.945066    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qnzv4\" (UniqueName: \"kubernetes.io/projected/7ab05096-862b-41f1-92ba-1eebceb2c026-kube-api-access-qnzv4\") pod \"e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-ch5zt\" (UID: \"7ab05096-862b-41f1-92ba-1eebceb2c026\") " pod="openshift-ingress/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-ch5zt"
Oct 30 22:02:23.945294 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:02:23.945113    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7ab05096-862b-41f1-92ba-1eebceb2c026-istio-token\") pod \"e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-ch5zt\" (UID: \"7ab05096-862b-41f1-92ba-1eebceb2c026\") " pod="openshift-ingress/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-ch5zt"
Oct 30 22:02:23.945294 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:02:23.945199    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7ab05096-862b-41f1-92ba-1eebceb2c026-workload-certs\") pod \"e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-ch5zt\" (UID: \"7ab05096-862b-41f1-92ba-1eebceb2c026\") " pod="openshift-ingress/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-ch5zt"
Oct 30 22:02:23.945294 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:02:23.945239    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7ab05096-862b-41f1-92ba-1eebceb2c026-istio-envoy\") pod \"e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-ch5zt\" (UID: \"7ab05096-862b-41f1-92ba-1eebceb2c026\") " pod="openshift-ingress/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-ch5zt"
Oct 30 22:02:23.945294 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:02:23.945279    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7ab05096-862b-41f1-92ba-1eebceb2c026-istio-data\") pod \"e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-ch5zt\" (UID: \"7ab05096-862b-41f1-92ba-1eebceb2c026\") " pod="openshift-ingress/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-ch5zt"
Oct 30 22:02:23.945464 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:02:23.945419    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7ab05096-862b-41f1-92ba-1eebceb2c026-istiod-ca-cert\") pod \"e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-ch5zt\" (UID: \"7ab05096-862b-41f1-92ba-1eebceb2c026\") " pod="openshift-ingress/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-ch5zt"
Oct 30 22:02:23.945528 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:02:23.945467    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7ab05096-862b-41f1-92ba-1eebceb2c026-istio-podinfo\") pod \"e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-ch5zt\" (UID: \"7ab05096-862b-41f1-92ba-1eebceb2c026\") " pod="openshift-ingress/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-ch5zt"
Oct 30 22:02:23.945528 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:02:23.945487    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ab05096-862b-41f1-92ba-1eebceb2c026-credential-socket\") pod \"e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-ch5zt\" (UID: \"7ab05096-862b-41f1-92ba-1eebceb2c026\") " pod="openshift-ingress/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-ch5zt"
Oct 30 22:02:23.945613 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:02:23.945595    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ab05096-862b-41f1-92ba-1eebceb2c026-workload-socket\") pod \"e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-ch5zt\" (UID: \"7ab05096-862b-41f1-92ba-1eebceb2c026\") " pod="openshift-ingress/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-ch5zt"
Oct 30 22:02:24.046860 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:02:24.046827    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ab05096-862b-41f1-92ba-1eebceb2c026-workload-socket\") pod \"e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-ch5zt\" (UID: \"7ab05096-862b-41f1-92ba-1eebceb2c026\") " pod="openshift-ingress/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-ch5zt"
Oct 30 22:02:24.047084 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:02:24.046888    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qnzv4\" (UniqueName: \"kubernetes.io/projected/7ab05096-862b-41f1-92ba-1eebceb2c026-kube-api-access-qnzv4\") pod \"e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-ch5zt\" (UID: \"7ab05096-862b-41f1-92ba-1eebceb2c026\") " pod="openshift-ingress/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-ch5zt"
Oct 30 22:02:24.047084 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:02:24.046910    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7ab05096-862b-41f1-92ba-1eebceb2c026-istio-token\") pod \"e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-ch5zt\" (UID: \"7ab05096-862b-41f1-92ba-1eebceb2c026\") " pod="openshift-ingress/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-ch5zt"
Oct 30 22:02:24.047084 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:02:24.046974    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7ab05096-862b-41f1-92ba-1eebceb2c026-workload-certs\") pod \"e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-ch5zt\" (UID: \"7ab05096-862b-41f1-92ba-1eebceb2c026\") " pod="openshift-ingress/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-ch5zt"
Oct 30 22:02:24.047084 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:02:24.046999    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7ab05096-862b-41f1-92ba-1eebceb2c026-istio-envoy\") pod \"e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-ch5zt\" (UID: \"7ab05096-862b-41f1-92ba-1eebceb2c026\") " pod="openshift-ingress/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-ch5zt"
Oct 30 22:02:24.047084 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:02:24.047017    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7ab05096-862b-41f1-92ba-1eebceb2c026-istio-data\") pod \"e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-ch5zt\" (UID: \"7ab05096-862b-41f1-92ba-1eebceb2c026\") " pod="openshift-ingress/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-ch5zt"
Oct 30 22:02:24.047310 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:02:24.047174    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7ab05096-862b-41f1-92ba-1eebceb2c026-istiod-ca-cert\") pod \"e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-ch5zt\" (UID: \"7ab05096-862b-41f1-92ba-1eebceb2c026\") " pod="openshift-ingress/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-ch5zt"
Oct 30 22:02:24.047310 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:02:24.047239    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7ab05096-862b-41f1-92ba-1eebceb2c026-istio-podinfo\") pod \"e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-ch5zt\" (UID: \"7ab05096-862b-41f1-92ba-1eebceb2c026\") " pod="openshift-ingress/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-ch5zt"
Oct 30 22:02:24.047310 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:02:24.047267    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ab05096-862b-41f1-92ba-1eebceb2c026-credential-socket\") pod \"e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-ch5zt\" (UID: \"7ab05096-862b-41f1-92ba-1eebceb2c026\") " pod="openshift-ingress/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-ch5zt"
Oct 30 22:02:24.047428 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:02:24.047318    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ab05096-862b-41f1-92ba-1eebceb2c026-workload-socket\") pod \"e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-ch5zt\" (UID: \"7ab05096-862b-41f1-92ba-1eebceb2c026\") " pod="openshift-ingress/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-ch5zt"
Oct 30 22:02:24.047727 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:02:24.047697    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ab05096-862b-41f1-92ba-1eebceb2c026-credential-socket\") pod \"e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-ch5zt\" (UID: \"7ab05096-862b-41f1-92ba-1eebceb2c026\") " pod="openshift-ingress/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-ch5zt"
Oct 30 22:02:24.047727 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:02:24.047721    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7ab05096-862b-41f1-92ba-1eebceb2c026-istio-data\") pod \"e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-ch5zt\" (UID: \"7ab05096-862b-41f1-92ba-1eebceb2c026\") " pod="openshift-ingress/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-ch5zt"
Oct 30 22:02:24.047936 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:02:24.047863    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7ab05096-862b-41f1-92ba-1eebceb2c026-istiod-ca-cert\") pod \"e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-ch5zt\" (UID: \"7ab05096-862b-41f1-92ba-1eebceb2c026\") " pod="openshift-ingress/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-ch5zt"
Oct 30 22:02:24.048009 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:02:24.047997    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7ab05096-862b-41f1-92ba-1eebceb2c026-workload-certs\") pod \"e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-ch5zt\" (UID: \"7ab05096-862b-41f1-92ba-1eebceb2c026\") " pod="openshift-ingress/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-ch5zt"
Oct 30 22:02:24.050457 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:02:24.050431    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7ab05096-862b-41f1-92ba-1eebceb2c026-istio-envoy\") pod \"e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-ch5zt\" (UID: \"7ab05096-862b-41f1-92ba-1eebceb2c026\") " pod="openshift-ingress/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-ch5zt"
Oct 30 22:02:24.050618 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:02:24.050603    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7ab05096-862b-41f1-92ba-1eebceb2c026-istio-podinfo\") pod \"e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-ch5zt\" (UID: \"7ab05096-862b-41f1-92ba-1eebceb2c026\") " pod="openshift-ingress/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-ch5zt"
Oct 30 22:02:24.082785 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:02:24.082747    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-ch5zt"]
Oct 30 22:02:24.105840 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:02:24.105794    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qnzv4\" (UniqueName: \"kubernetes.io/projected/7ab05096-862b-41f1-92ba-1eebceb2c026-kube-api-access-qnzv4\") pod \"e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-ch5zt\" (UID: \"7ab05096-862b-41f1-92ba-1eebceb2c026\") " pod="openshift-ingress/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-ch5zt"
Oct 30 22:02:24.106069 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:02:24.105903    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7ab05096-862b-41f1-92ba-1eebceb2c026-istio-token\") pod \"e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-ch5zt\" (UID: \"7ab05096-862b-41f1-92ba-1eebceb2c026\") " pod="openshift-ingress/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-ch5zt"
Oct 30 22:02:24.187061 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:02:24.186967    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-ch5zt"
Oct 30 22:02:24.620748 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:02:24.620713    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-ch5zt"]
Oct 30 22:02:24.621752 ip-10-0-140-18 kubenswrapper[4473]: W1030 22:02:24.621725    4473 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ab05096_862b_41f1_92ba_1eebceb2c026.slice/crio-e1a9c90b5bdec133574d00947aa27eb3f87dcb0edb3f5f6f1db36ba0397a703b WatchSource:0}: Error finding container e1a9c90b5bdec133574d00947aa27eb3f87dcb0edb3f5f6f1db36ba0397a703b: Status 404 returned error can't find the container with id e1a9c90b5bdec133574d00947aa27eb3f87dcb0edb3f5f6f1db36ba0397a703b
Oct 30 22:02:24.624269 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:02:24.624234    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:02:24.624362 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:02:24.624305    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:02:24.624362 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:02:24.624340    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:02:24.798584 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:02:24.798018    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-ch5zt" event={"ID":"7ab05096-862b-41f1-92ba-1eebceb2c026","Type":"ContainerStarted","Data":"0cb61c146d53f60d49f563776d31c3bd873a94fbc41466dc3bde1ab6ab8f130e"}
Oct 30 22:02:24.798584 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:02:24.798142    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-ch5zt" event={"ID":"7ab05096-862b-41f1-92ba-1eebceb2c026","Type":"ContainerStarted","Data":"e1a9c90b5bdec133574d00947aa27eb3f87dcb0edb3f5f6f1db36ba0397a703b"}
Oct 30 22:02:24.922992 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:02:24.922867    4473 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-ch5zt" podStartSLOduration=1.92285295 podStartE2EDuration="1.92285295s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:02:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:02:24.919437551 +0000 UTC m=+1190.692806064" watchObservedRunningTime="2026-10-30 22:02:24.92285295 +0000 UTC m=+1190.696221509"
Oct 30 22:02:25.187541 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:02:25.187449    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-ch5zt"
Oct 30 22:02:25.189143 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:02:25.189119    4473 patch_prober.go:29] interesting pod/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-ch5zt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused" start-of-body=
Oct 30 22:02:25.189265 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:02:25.189174    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-ch5zt" podUID="7ab05096-862b-41f1-92ba-1eebceb2c026" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused"
Oct 30 22:02:26.187612 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:02:26.187577    4473 patch_prober.go:29] interesting pod/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-ch5zt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused" start-of-body=
Oct 30 22:02:26.188072 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:02:26.187637    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-ch5zt" podUID="7ab05096-862b-41f1-92ba-1eebceb2c026" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused"
Oct 30 22:02:27.187816 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:02:27.187776    4473 patch_prober.go:29] interesting pod/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-ch5zt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused" start-of-body=
Oct 30 22:02:27.188392 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:02:27.187830    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-ch5zt" podUID="7ab05096-862b-41f1-92ba-1eebceb2c026" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused"
Oct 30 22:02:28.188332 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:02:28.188297    4473 patch_prober.go:29] interesting pod/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-ch5zt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused" start-of-body=
Oct 30 22:02:28.188732 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:02:28.188351    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-ch5zt" podUID="7ab05096-862b-41f1-92ba-1eebceb2c026" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused"
Oct 30 22:02:29.187359 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:02:29.187320    4473 patch_prober.go:29] interesting pod/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-ch5zt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused" start-of-body=
Oct 30 22:02:29.187541 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:02:29.187388    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-ch5zt" podUID="7ab05096-862b-41f1-92ba-1eebceb2c026" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused"
Oct 30 22:02:30.196945 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:02:30.187695    4473 patch_prober.go:29] interesting pod/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-ch5zt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused" start-of-body=
Oct 30 22:02:30.196945 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:02:30.187776    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-ch5zt" podUID="7ab05096-862b-41f1-92ba-1eebceb2c026" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused"
Oct 30 22:02:31.187657 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:02:31.187618    4473 patch_prober.go:29] interesting pod/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-ch5zt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused" start-of-body=
Oct 30 22:02:31.187872 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:02:31.187683    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-ch5zt" podUID="7ab05096-862b-41f1-92ba-1eebceb2c026" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused"
Oct 30 22:02:32.209420 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:02:32.209385    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-ch5zt"
Oct 30 22:02:32.211050 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:02:32.210975    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-ch5zt"
Oct 30 22:02:32.211835 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:02:32.211810    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-ch5zt"
Oct 30 22:02:33.905039 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:02:33.905003    4473 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-dde1e8/distinct-2-w0-dde1e8-kserve-744857d57-ftt8b"]
Oct 30 22:02:33.909378 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:02:33.909353    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-dde1e8/distinct-2-w0-dde1e8-kserve-744857d57-ftt8b"
Oct 30 22:02:34.010226 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:02:34.010182    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/577b020d-2a6b-4020-863c-cf19d4f0ca3b-home\") pod \"distinct-2-w0-dde1e8-kserve-744857d57-ftt8b\" (UID: \"577b020d-2a6b-4020-863c-cf19d4f0ca3b\") " pod="e2e-models-e2e-worker-w0-dde1e8/distinct-2-w0-dde1e8-kserve-744857d57-ftt8b"
Oct 30 22:02:34.010434 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:02:34.010369    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/577b020d-2a6b-4020-863c-cf19d4f0ca3b-model-cache\") pod \"distinct-2-w0-dde1e8-kserve-744857d57-ftt8b\" (UID: \"577b020d-2a6b-4020-863c-cf19d4f0ca3b\") " pod="e2e-models-e2e-worker-w0-dde1e8/distinct-2-w0-dde1e8-kserve-744857d57-ftt8b"
Oct 30 22:02:34.010659 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:02:34.010639    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/577b020d-2a6b-4020-863c-cf19d4f0ca3b-tmp-dir\") pod \"distinct-2-w0-dde1e8-kserve-744857d57-ftt8b\" (UID: \"577b020d-2a6b-4020-863c-cf19d4f0ca3b\") " pod="e2e-models-e2e-worker-w0-dde1e8/distinct-2-w0-dde1e8-kserve-744857d57-ftt8b"
Oct 30 22:02:34.010707 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:02:34.010675    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/577b020d-2a6b-4020-863c-cf19d4f0ca3b-tls-certs\") pod \"distinct-2-w0-dde1e8-kserve-744857d57-ftt8b\" (UID: \"577b020d-2a6b-4020-863c-cf19d4f0ca3b\") " pod="e2e-models-e2e-worker-w0-dde1e8/distinct-2-w0-dde1e8-kserve-744857d57-ftt8b"
Oct 30 22:02:34.010914 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:02:34.010898    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qlhjc\" (UniqueName: \"kubernetes.io/projected/577b020d-2a6b-4020-863c-cf19d4f0ca3b-kube-api-access-qlhjc\") pod \"distinct-2-w0-dde1e8-kserve-744857d57-ftt8b\" (UID: \"577b020d-2a6b-4020-863c-cf19d4f0ca3b\") " pod="e2e-models-e2e-worker-w0-dde1e8/distinct-2-w0-dde1e8-kserve-744857d57-ftt8b"
Oct 30 22:02:34.011042 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:02:34.011028    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/577b020d-2a6b-4020-863c-cf19d4f0ca3b-dshm\") pod \"distinct-2-w0-dde1e8-kserve-744857d57-ftt8b\" (UID: \"577b020d-2a6b-4020-863c-cf19d4f0ca3b\") " pod="e2e-models-e2e-worker-w0-dde1e8/distinct-2-w0-dde1e8-kserve-744857d57-ftt8b"
Oct 30 22:02:34.018789 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:02:34.018746    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-dde1e8/distinct-2-w0-dde1e8-kserve-744857d57-ftt8b"]
Oct 30 22:02:34.019007 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:02:34.018989    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-dde1e8\"/\"distinct-2-w0-dde1e8-kserve-self-signed-certs\""
Oct 30 22:02:34.122954 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:02:34.122822    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/577b020d-2a6b-4020-863c-cf19d4f0ca3b-tls-certs\") pod \"distinct-2-w0-dde1e8-kserve-744857d57-ftt8b\" (UID: \"577b020d-2a6b-4020-863c-cf19d4f0ca3b\") " pod="e2e-models-e2e-worker-w0-dde1e8/distinct-2-w0-dde1e8-kserve-744857d57-ftt8b"
Oct 30 22:02:34.123152 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:02:34.123033    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qlhjc\" (UniqueName: \"kubernetes.io/projected/577b020d-2a6b-4020-863c-cf19d4f0ca3b-kube-api-access-qlhjc\") pod \"distinct-2-w0-dde1e8-kserve-744857d57-ftt8b\" (UID: \"577b020d-2a6b-4020-863c-cf19d4f0ca3b\") " pod="e2e-models-e2e-worker-w0-dde1e8/distinct-2-w0-dde1e8-kserve-744857d57-ftt8b"
Oct 30 22:02:34.123152 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:02:34.123097    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/577b020d-2a6b-4020-863c-cf19d4f0ca3b-dshm\") pod \"distinct-2-w0-dde1e8-kserve-744857d57-ftt8b\" (UID: \"577b020d-2a6b-4020-863c-cf19d4f0ca3b\") " pod="e2e-models-e2e-worker-w0-dde1e8/distinct-2-w0-dde1e8-kserve-744857d57-ftt8b"
Oct 30 22:02:34.123299 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:02:34.123200    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/577b020d-2a6b-4020-863c-cf19d4f0ca3b-home\") pod \"distinct-2-w0-dde1e8-kserve-744857d57-ftt8b\" (UID: \"577b020d-2a6b-4020-863c-cf19d4f0ca3b\") " pod="e2e-models-e2e-worker-w0-dde1e8/distinct-2-w0-dde1e8-kserve-744857d57-ftt8b"
Oct 30 22:02:34.123355 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:02:34.123306    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/577b020d-2a6b-4020-863c-cf19d4f0ca3b-model-cache\") pod \"distinct-2-w0-dde1e8-kserve-744857d57-ftt8b\" (UID: \"577b020d-2a6b-4020-863c-cf19d4f0ca3b\") " pod="e2e-models-e2e-worker-w0-dde1e8/distinct-2-w0-dde1e8-kserve-744857d57-ftt8b"
Oct 30 22:02:34.123409 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:02:34.123368    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/577b020d-2a6b-4020-863c-cf19d4f0ca3b-tmp-dir\") pod \"distinct-2-w0-dde1e8-kserve-744857d57-ftt8b\" (UID: \"577b020d-2a6b-4020-863c-cf19d4f0ca3b\") " pod="e2e-models-e2e-worker-w0-dde1e8/distinct-2-w0-dde1e8-kserve-744857d57-ftt8b"
Oct 30 22:02:34.126955 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:02:34.123894    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/577b020d-2a6b-4020-863c-cf19d4f0ca3b-tmp-dir\") pod \"distinct-2-w0-dde1e8-kserve-744857d57-ftt8b\" (UID: \"577b020d-2a6b-4020-863c-cf19d4f0ca3b\") " pod="e2e-models-e2e-worker-w0-dde1e8/distinct-2-w0-dde1e8-kserve-744857d57-ftt8b"
Oct 30 22:02:34.126955 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:02:34.125133    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/577b020d-2a6b-4020-863c-cf19d4f0ca3b-home\") pod \"distinct-2-w0-dde1e8-kserve-744857d57-ftt8b\" (UID: \"577b020d-2a6b-4020-863c-cf19d4f0ca3b\") " pod="e2e-models-e2e-worker-w0-dde1e8/distinct-2-w0-dde1e8-kserve-744857d57-ftt8b"
Oct 30 22:02:34.126955 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:02:34.125409    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/577b020d-2a6b-4020-863c-cf19d4f0ca3b-model-cache\") pod \"distinct-2-w0-dde1e8-kserve-744857d57-ftt8b\" (UID: \"577b020d-2a6b-4020-863c-cf19d4f0ca3b\") " pod="e2e-models-e2e-worker-w0-dde1e8/distinct-2-w0-dde1e8-kserve-744857d57-ftt8b"
Oct 30 22:02:34.138062 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:02:34.138026    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/577b020d-2a6b-4020-863c-cf19d4f0ca3b-dshm\") pod \"distinct-2-w0-dde1e8-kserve-744857d57-ftt8b\" (UID: \"577b020d-2a6b-4020-863c-cf19d4f0ca3b\") " pod="e2e-models-e2e-worker-w0-dde1e8/distinct-2-w0-dde1e8-kserve-744857d57-ftt8b"
Oct 30 22:02:34.144016 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:02:34.143983    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/577b020d-2a6b-4020-863c-cf19d4f0ca3b-tls-certs\") pod \"distinct-2-w0-dde1e8-kserve-744857d57-ftt8b\" (UID: \"577b020d-2a6b-4020-863c-cf19d4f0ca3b\") " pod="e2e-models-e2e-worker-w0-dde1e8/distinct-2-w0-dde1e8-kserve-744857d57-ftt8b"
Oct 30 22:02:34.183445 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:02:34.183368    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qlhjc\" (UniqueName: \"kubernetes.io/projected/577b020d-2a6b-4020-863c-cf19d4f0ca3b-kube-api-access-qlhjc\") pod \"distinct-2-w0-dde1e8-kserve-744857d57-ftt8b\" (UID: \"577b020d-2a6b-4020-863c-cf19d4f0ca3b\") " pod="e2e-models-e2e-worker-w0-dde1e8/distinct-2-w0-dde1e8-kserve-744857d57-ftt8b"
Oct 30 22:02:34.223749 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:02:34.223713    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-dde1e8/distinct-2-w0-dde1e8-kserve-744857d57-ftt8b"
Oct 30 22:02:34.812131 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:02:34.812100    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-dde1e8/distinct-2-w0-dde1e8-kserve-744857d57-ftt8b"]
Oct 30 22:02:34.812267 ip-10-0-140-18 kubenswrapper[4473]: W1030 22:02:34.812247    4473 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod577b020d_2a6b_4020_863c_cf19d4f0ca3b.slice/crio-223053fa58ab5cc28c870d9b12d92317c41740a6e04ca701d68f98e0b3fc9d7d WatchSource:0}: Error finding container 223053fa58ab5cc28c870d9b12d92317c41740a6e04ca701d68f98e0b3fc9d7d: Status 404 returned error can't find the container with id 223053fa58ab5cc28c870d9b12d92317c41740a6e04ca701d68f98e0b3fc9d7d
Oct 30 22:02:34.895026 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:02:34.894984    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-dde1e8/distinct-2-w0-dde1e8-kserve-744857d57-ftt8b" event={"ID":"577b020d-2a6b-4020-863c-cf19d4f0ca3b","Type":"ContainerStarted","Data":"9767ba5bd731c4c231fd2880310ee1358a01dce2e52aad070d6a7aca6f695cb7"}
Oct 30 22:02:34.895237 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:02:34.895043    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-dde1e8/distinct-2-w0-dde1e8-kserve-744857d57-ftt8b" event={"ID":"577b020d-2a6b-4020-863c-cf19d4f0ca3b","Type":"ContainerStarted","Data":"223053fa58ab5cc28c870d9b12d92317c41740a6e04ca701d68f98e0b3fc9d7d"}
Oct 30 22:02:35.013451 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:02:35.013392    4473 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-dde1e8/distinct-2-w0-dde1e8-kserve-744857d57-ftt8b" podStartSLOduration=2.013378587 podStartE2EDuration="2.013378587s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:02:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:02:35.011977455 +0000 UTC m=+1200.785345942" watchObservedRunningTime="2026-10-30 22:02:35.013378587 +0000 UTC m=+1200.786747074"
Oct 30 22:02:35.244004 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:02:35.237025    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-dde1e8/distinct-2-w0-dde1e8-kserve-744857d57-ftt8b"
Oct 30 22:02:44.224470 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:02:44.224437    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-dde1e8/distinct-2-w0-dde1e8-kserve-744857d57-ftt8b"
Oct 30 22:02:44.237770 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:02:44.237743    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-dde1e8/distinct-2-w0-dde1e8-kserve-744857d57-ftt8b"
Oct 30 22:02:44.995177 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:02:44.995081    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-dde1e8/distinct-2-w0-dde1e8-kserve-744857d57-ftt8b"
Oct 30 22:02:50.326524 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:02:50.326489    4473 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fncvg" containerName="extractor"
Oct 30 22:03:00.537700 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:00.537659    4473 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w3-0f42cb/distinct-2-w3-0f42cb-kserve-fc477897b-jk8qc"]
Oct 30 22:03:00.542313 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:00.542287    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-0f42cb/distinct-2-w3-0f42cb-kserve-fc477897b-jk8qc"
Oct 30 22:03:00.596321 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:00.596003    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-0f42cb\"/\"distinct-2-w3-0f42cb-kserve-self-signed-certs\""
Oct 30 22:03:00.649198 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:00.649150    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-0f42cb/distinct-2-w3-0f42cb-kserve-fc477897b-jk8qc"]
Oct 30 22:03:00.717791 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:00.717747    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/61b49182-057e-41af-b609-1a6abf82c197-tls-certs\") pod \"distinct-2-w3-0f42cb-kserve-fc477897b-jk8qc\" (UID: \"61b49182-057e-41af-b609-1a6abf82c197\") " pod="e2e-models-e2e-worker-w3-0f42cb/distinct-2-w3-0f42cb-kserve-fc477897b-jk8qc"
Oct 30 22:03:00.718184 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:00.718162    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wtn7p\" (UniqueName: \"kubernetes.io/projected/61b49182-057e-41af-b609-1a6abf82c197-kube-api-access-wtn7p\") pod \"distinct-2-w3-0f42cb-kserve-fc477897b-jk8qc\" (UID: \"61b49182-057e-41af-b609-1a6abf82c197\") " pod="e2e-models-e2e-worker-w3-0f42cb/distinct-2-w3-0f42cb-kserve-fc477897b-jk8qc"
Oct 30 22:03:00.718732 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:00.718710    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/61b49182-057e-41af-b609-1a6abf82c197-dshm\") pod \"distinct-2-w3-0f42cb-kserve-fc477897b-jk8qc\" (UID: \"61b49182-057e-41af-b609-1a6abf82c197\") " pod="e2e-models-e2e-worker-w3-0f42cb/distinct-2-w3-0f42cb-kserve-fc477897b-jk8qc"
Oct 30 22:03:00.718847 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:00.718833    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/61b49182-057e-41af-b609-1a6abf82c197-home\") pod \"distinct-2-w3-0f42cb-kserve-fc477897b-jk8qc\" (UID: \"61b49182-057e-41af-b609-1a6abf82c197\") " pod="e2e-models-e2e-worker-w3-0f42cb/distinct-2-w3-0f42cb-kserve-fc477897b-jk8qc"
Oct 30 22:03:00.719017 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:00.719003    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/61b49182-057e-41af-b609-1a6abf82c197-model-cache\") pod \"distinct-2-w3-0f42cb-kserve-fc477897b-jk8qc\" (UID: \"61b49182-057e-41af-b609-1a6abf82c197\") " pod="e2e-models-e2e-worker-w3-0f42cb/distinct-2-w3-0f42cb-kserve-fc477897b-jk8qc"
Oct 30 22:03:00.719253 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:00.719236    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/61b49182-057e-41af-b609-1a6abf82c197-tmp-dir\") pod \"distinct-2-w3-0f42cb-kserve-fc477897b-jk8qc\" (UID: \"61b49182-057e-41af-b609-1a6abf82c197\") " pod="e2e-models-e2e-worker-w3-0f42cb/distinct-2-w3-0f42cb-kserve-fc477897b-jk8qc"
Oct 30 22:03:00.820358 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:00.820273    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/61b49182-057e-41af-b609-1a6abf82c197-tls-certs\") pod \"distinct-2-w3-0f42cb-kserve-fc477897b-jk8qc\" (UID: \"61b49182-057e-41af-b609-1a6abf82c197\") " pod="e2e-models-e2e-worker-w3-0f42cb/distinct-2-w3-0f42cb-kserve-fc477897b-jk8qc"
Oct 30 22:03:00.820519 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:00.820364    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wtn7p\" (UniqueName: \"kubernetes.io/projected/61b49182-057e-41af-b609-1a6abf82c197-kube-api-access-wtn7p\") pod \"distinct-2-w3-0f42cb-kserve-fc477897b-jk8qc\" (UID: \"61b49182-057e-41af-b609-1a6abf82c197\") " pod="e2e-models-e2e-worker-w3-0f42cb/distinct-2-w3-0f42cb-kserve-fc477897b-jk8qc"
Oct 30 22:03:00.820519 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:00.820453    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/61b49182-057e-41af-b609-1a6abf82c197-dshm\") pod \"distinct-2-w3-0f42cb-kserve-fc477897b-jk8qc\" (UID: \"61b49182-057e-41af-b609-1a6abf82c197\") " pod="e2e-models-e2e-worker-w3-0f42cb/distinct-2-w3-0f42cb-kserve-fc477897b-jk8qc"
Oct 30 22:03:00.820519 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:00.820473    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/61b49182-057e-41af-b609-1a6abf82c197-home\") pod \"distinct-2-w3-0f42cb-kserve-fc477897b-jk8qc\" (UID: \"61b49182-057e-41af-b609-1a6abf82c197\") " pod="e2e-models-e2e-worker-w3-0f42cb/distinct-2-w3-0f42cb-kserve-fc477897b-jk8qc"
Oct 30 22:03:00.820678 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:00.820523    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/61b49182-057e-41af-b609-1a6abf82c197-model-cache\") pod \"distinct-2-w3-0f42cb-kserve-fc477897b-jk8qc\" (UID: \"61b49182-057e-41af-b609-1a6abf82c197\") " pod="e2e-models-e2e-worker-w3-0f42cb/distinct-2-w3-0f42cb-kserve-fc477897b-jk8qc"
Oct 30 22:03:00.820678 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:00.820554    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/61b49182-057e-41af-b609-1a6abf82c197-tmp-dir\") pod \"distinct-2-w3-0f42cb-kserve-fc477897b-jk8qc\" (UID: \"61b49182-057e-41af-b609-1a6abf82c197\") " pod="e2e-models-e2e-worker-w3-0f42cb/distinct-2-w3-0f42cb-kserve-fc477897b-jk8qc"
Oct 30 22:03:00.820968 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:00.820945    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/61b49182-057e-41af-b609-1a6abf82c197-home\") pod \"distinct-2-w3-0f42cb-kserve-fc477897b-jk8qc\" (UID: \"61b49182-057e-41af-b609-1a6abf82c197\") " pod="e2e-models-e2e-worker-w3-0f42cb/distinct-2-w3-0f42cb-kserve-fc477897b-jk8qc"
Oct 30 22:03:00.821043 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:00.820982    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/61b49182-057e-41af-b609-1a6abf82c197-model-cache\") pod \"distinct-2-w3-0f42cb-kserve-fc477897b-jk8qc\" (UID: \"61b49182-057e-41af-b609-1a6abf82c197\") " pod="e2e-models-e2e-worker-w3-0f42cb/distinct-2-w3-0f42cb-kserve-fc477897b-jk8qc"
Oct 30 22:03:00.821092 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:00.821063    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/61b49182-057e-41af-b609-1a6abf82c197-tmp-dir\") pod \"distinct-2-w3-0f42cb-kserve-fc477897b-jk8qc\" (UID: \"61b49182-057e-41af-b609-1a6abf82c197\") " pod="e2e-models-e2e-worker-w3-0f42cb/distinct-2-w3-0f42cb-kserve-fc477897b-jk8qc"
Oct 30 22:03:00.823018 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:00.822996    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/61b49182-057e-41af-b609-1a6abf82c197-dshm\") pod \"distinct-2-w3-0f42cb-kserve-fc477897b-jk8qc\" (UID: \"61b49182-057e-41af-b609-1a6abf82c197\") " pod="e2e-models-e2e-worker-w3-0f42cb/distinct-2-w3-0f42cb-kserve-fc477897b-jk8qc"
Oct 30 22:03:00.823218 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:00.823200    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/61b49182-057e-41af-b609-1a6abf82c197-tls-certs\") pod \"distinct-2-w3-0f42cb-kserve-fc477897b-jk8qc\" (UID: \"61b49182-057e-41af-b609-1a6abf82c197\") " pod="e2e-models-e2e-worker-w3-0f42cb/distinct-2-w3-0f42cb-kserve-fc477897b-jk8qc"
Oct 30 22:03:00.881724 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:00.881690    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wtn7p\" (UniqueName: \"kubernetes.io/projected/61b49182-057e-41af-b609-1a6abf82c197-kube-api-access-wtn7p\") pod \"distinct-2-w3-0f42cb-kserve-fc477897b-jk8qc\" (UID: \"61b49182-057e-41af-b609-1a6abf82c197\") " pod="e2e-models-e2e-worker-w3-0f42cb/distinct-2-w3-0f42cb-kserve-fc477897b-jk8qc"
Oct 30 22:03:01.156618 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:01.156583    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-0f42cb/distinct-2-w3-0f42cb-kserve-fc477897b-jk8qc"
Oct 30 22:03:01.536877 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:01.536836    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-0f42cb/distinct-2-w3-0f42cb-kserve-fc477897b-jk8qc"]
Oct 30 22:03:01.537326 ip-10-0-140-18 kubenswrapper[4473]: W1030 22:03:01.537305    4473 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod61b49182_057e_41af_b609_1a6abf82c197.slice/crio-44ebd893326f750559874906f4a3a722bb4739eb8f1223c5b3923b74c58a2c07 WatchSource:0}: Error finding container 44ebd893326f750559874906f4a3a722bb4739eb8f1223c5b3923b74c58a2c07: Status 404 returned error can't find the container with id 44ebd893326f750559874906f4a3a722bb4739eb8f1223c5b3923b74c58a2c07
Oct 30 22:03:02.142096 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:02.142055    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-0f42cb/distinct-2-w3-0f42cb-kserve-fc477897b-jk8qc" event={"ID":"61b49182-057e-41af-b609-1a6abf82c197","Type":"ContainerStarted","Data":"15fa7e928582a0d68845747079e81169d32cd3c005241a40730b346d274eb1df"}
Oct 30 22:03:02.142505 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:02.142110    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-0f42cb/distinct-2-w3-0f42cb-kserve-fc477897b-jk8qc" event={"ID":"61b49182-057e-41af-b609-1a6abf82c197","Type":"ContainerStarted","Data":"44ebd893326f750559874906f4a3a722bb4739eb8f1223c5b3923b74c58a2c07"}
Oct 30 22:03:02.157689 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:02.157641    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-0f42cb/distinct-2-w3-0f42cb-kserve-fc477897b-jk8qc"
Oct 30 22:03:02.262794 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:02.262734    4473 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w3-0f42cb/distinct-2-w3-0f42cb-kserve-fc477897b-jk8qc" podStartSLOduration=2.262719547 podStartE2EDuration="2.262719547s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:03:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:03:02.259639819 +0000 UTC m=+1228.033008310" watchObservedRunningTime="2026-10-30 22:03:02.262719547 +0000 UTC m=+1228.036088064"
Oct 30 22:03:09.433380 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:09.433343    4473 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-5qn5b"]
Oct 30 22:03:09.437955 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:09.437913    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-5qn5b"
Oct 30 22:03:09.563506 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:09.563472    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/72b2d008-08fc-41aa-9515-ad4319bee895-workload-certs\") pod \"e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-5qn5b\" (UID: \"72b2d008-08fc-41aa-9515-ad4319bee895\") " pod="openshift-ingress/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-5qn5b"
Oct 30 22:03:09.563735 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:09.563705    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/72b2d008-08fc-41aa-9515-ad4319bee895-istio-data\") pod \"e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-5qn5b\" (UID: \"72b2d008-08fc-41aa-9515-ad4319bee895\") " pod="openshift-ingress/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-5qn5b"
Oct 30 22:03:09.563884 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:09.563805    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/72b2d008-08fc-41aa-9515-ad4319bee895-istio-podinfo\") pod \"e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-5qn5b\" (UID: \"72b2d008-08fc-41aa-9515-ad4319bee895\") " pod="openshift-ingress/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-5qn5b"
Oct 30 22:03:09.565218 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:09.563881    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/72b2d008-08fc-41aa-9515-ad4319bee895-istio-token\") pod \"e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-5qn5b\" (UID: \"72b2d008-08fc-41aa-9515-ad4319bee895\") " pod="openshift-ingress/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-5qn5b"
Oct 30 22:03:09.565218 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:09.563943    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/72b2d008-08fc-41aa-9515-ad4319bee895-credential-socket\") pod \"e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-5qn5b\" (UID: \"72b2d008-08fc-41aa-9515-ad4319bee895\") " pod="openshift-ingress/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-5qn5b"
Oct 30 22:03:09.565218 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:09.564017    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/72b2d008-08fc-41aa-9515-ad4319bee895-istio-envoy\") pod \"e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-5qn5b\" (UID: \"72b2d008-08fc-41aa-9515-ad4319bee895\") " pod="openshift-ingress/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-5qn5b"
Oct 30 22:03:09.565218 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:09.564192    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/72b2d008-08fc-41aa-9515-ad4319bee895-istiod-ca-cert\") pod \"e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-5qn5b\" (UID: \"72b2d008-08fc-41aa-9515-ad4319bee895\") " pod="openshift-ingress/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-5qn5b"
Oct 30 22:03:09.565218 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:09.564269    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/72b2d008-08fc-41aa-9515-ad4319bee895-workload-socket\") pod \"e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-5qn5b\" (UID: \"72b2d008-08fc-41aa-9515-ad4319bee895\") " pod="openshift-ingress/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-5qn5b"
Oct 30 22:03:09.565218 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:09.564343    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j4rsj\" (UniqueName: \"kubernetes.io/projected/72b2d008-08fc-41aa-9515-ad4319bee895-kube-api-access-j4rsj\") pod \"e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-5qn5b\" (UID: \"72b2d008-08fc-41aa-9515-ad4319bee895\") " pod="openshift-ingress/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-5qn5b"
Oct 30 22:03:09.639796 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:09.639759    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-5qn5b"]
Oct 30 22:03:09.665770 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:09.665729    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/72b2d008-08fc-41aa-9515-ad4319bee895-istio-envoy\") pod \"e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-5qn5b\" (UID: \"72b2d008-08fc-41aa-9515-ad4319bee895\") " pod="openshift-ingress/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-5qn5b"
Oct 30 22:03:09.665987 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:09.665810    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/72b2d008-08fc-41aa-9515-ad4319bee895-istiod-ca-cert\") pod \"e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-5qn5b\" (UID: \"72b2d008-08fc-41aa-9515-ad4319bee895\") " pod="openshift-ingress/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-5qn5b"
Oct 30 22:03:09.665987 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:09.665868    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/72b2d008-08fc-41aa-9515-ad4319bee895-workload-socket\") pod \"e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-5qn5b\" (UID: \"72b2d008-08fc-41aa-9515-ad4319bee895\") " pod="openshift-ingress/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-5qn5b"
Oct 30 22:03:09.665987 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:09.665964    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j4rsj\" (UniqueName: \"kubernetes.io/projected/72b2d008-08fc-41aa-9515-ad4319bee895-kube-api-access-j4rsj\") pod \"e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-5qn5b\" (UID: \"72b2d008-08fc-41aa-9515-ad4319bee895\") " pod="openshift-ingress/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-5qn5b"
Oct 30 22:03:09.666173 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:09.666132    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/72b2d008-08fc-41aa-9515-ad4319bee895-workload-certs\") pod \"e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-5qn5b\" (UID: \"72b2d008-08fc-41aa-9515-ad4319bee895\") " pod="openshift-ingress/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-5qn5b"
Oct 30 22:03:09.666173 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:09.666157    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/72b2d008-08fc-41aa-9515-ad4319bee895-istio-data\") pod \"e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-5qn5b\" (UID: \"72b2d008-08fc-41aa-9515-ad4319bee895\") " pod="openshift-ingress/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-5qn5b"
Oct 30 22:03:09.666270 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:09.666192    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/72b2d008-08fc-41aa-9515-ad4319bee895-istio-podinfo\") pod \"e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-5qn5b\" (UID: \"72b2d008-08fc-41aa-9515-ad4319bee895\") " pod="openshift-ingress/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-5qn5b"
Oct 30 22:03:09.666270 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:09.666257    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/72b2d008-08fc-41aa-9515-ad4319bee895-istio-token\") pod \"e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-5qn5b\" (UID: \"72b2d008-08fc-41aa-9515-ad4319bee895\") " pod="openshift-ingress/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-5qn5b"
Oct 30 22:03:09.666438 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:09.666300    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/72b2d008-08fc-41aa-9515-ad4319bee895-credential-socket\") pod \"e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-5qn5b\" (UID: \"72b2d008-08fc-41aa-9515-ad4319bee895\") " pod="openshift-ingress/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-5qn5b"
Oct 30 22:03:09.666746 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:09.666642    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/72b2d008-08fc-41aa-9515-ad4319bee895-istio-data\") pod \"e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-5qn5b\" (UID: \"72b2d008-08fc-41aa-9515-ad4319bee895\") " pod="openshift-ingress/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-5qn5b"
Oct 30 22:03:09.666865 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:09.666830    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/72b2d008-08fc-41aa-9515-ad4319bee895-credential-socket\") pod \"e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-5qn5b\" (UID: \"72b2d008-08fc-41aa-9515-ad4319bee895\") " pod="openshift-ingress/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-5qn5b"
Oct 30 22:03:09.666865 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:09.666852    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/72b2d008-08fc-41aa-9515-ad4319bee895-istiod-ca-cert\") pod \"e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-5qn5b\" (UID: \"72b2d008-08fc-41aa-9515-ad4319bee895\") " pod="openshift-ingress/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-5qn5b"
Oct 30 22:03:09.667182 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:09.666955    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/72b2d008-08fc-41aa-9515-ad4319bee895-workload-socket\") pod \"e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-5qn5b\" (UID: \"72b2d008-08fc-41aa-9515-ad4319bee895\") " pod="openshift-ingress/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-5qn5b"
Oct 30 22:03:09.667681 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:09.667621    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/72b2d008-08fc-41aa-9515-ad4319bee895-workload-certs\") pod \"e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-5qn5b\" (UID: \"72b2d008-08fc-41aa-9515-ad4319bee895\") " pod="openshift-ingress/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-5qn5b"
Oct 30 22:03:09.669563 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:09.669510    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/72b2d008-08fc-41aa-9515-ad4319bee895-istio-envoy\") pod \"e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-5qn5b\" (UID: \"72b2d008-08fc-41aa-9515-ad4319bee895\") " pod="openshift-ingress/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-5qn5b"
Oct 30 22:03:09.669908 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:09.669888    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/72b2d008-08fc-41aa-9515-ad4319bee895-istio-podinfo\") pod \"e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-5qn5b\" (UID: \"72b2d008-08fc-41aa-9515-ad4319bee895\") " pod="openshift-ingress/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-5qn5b"
Oct 30 22:03:09.724746 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:09.724671    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/72b2d008-08fc-41aa-9515-ad4319bee895-istio-token\") pod \"e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-5qn5b\" (UID: \"72b2d008-08fc-41aa-9515-ad4319bee895\") " pod="openshift-ingress/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-5qn5b"
Oct 30 22:03:09.724996 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:09.724975    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j4rsj\" (UniqueName: \"kubernetes.io/projected/72b2d008-08fc-41aa-9515-ad4319bee895-kube-api-access-j4rsj\") pod \"e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-5qn5b\" (UID: \"72b2d008-08fc-41aa-9515-ad4319bee895\") " pod="openshift-ingress/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-5qn5b"
Oct 30 22:03:09.753198 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:09.753168    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-5qn5b"
Oct 30 22:03:10.222382 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:10.222352    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-5qn5b"]
Oct 30 22:03:10.223721 ip-10-0-140-18 kubenswrapper[4473]: W1030 22:03:10.223693    4473 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72b2d008_08fc_41aa_9515_ad4319bee895.slice/crio-392fd41853a153eeb0c0047fe98bb58e2b3eadacec562ed9c6aaf542c6f9efca WatchSource:0}: Error finding container 392fd41853a153eeb0c0047fe98bb58e2b3eadacec562ed9c6aaf542c6f9efca: Status 404 returned error can't find the container with id 392fd41853a153eeb0c0047fe98bb58e2b3eadacec562ed9c6aaf542c6f9efca
Oct 30 22:03:10.226309 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:10.226270    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:03:10.226378 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:10.226341    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:03:10.226378 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:10.226369    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:03:10.240216 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:10.240183    4473 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w2-0c99b1/distinct-2-w2-0c99b1-kserve-7466864658-8z7fk"]
Oct 30 22:03:10.246061 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:10.246024    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-0c99b1/distinct-2-w2-0c99b1-kserve-7466864658-8z7fk"
Oct 30 22:03:10.299390 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:10.299364    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w2-0c99b1\"/\"distinct-2-w2-0c99b1-kserve-self-signed-certs\""
Oct 30 22:03:10.299568 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:10.299513    4473 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w2-0c99b1\"/\"kube-root-ca.crt\""
Oct 30 22:03:10.301655 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:10.301631    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-0c99b1/distinct-2-w2-0c99b1-kserve-7466864658-8z7fk"]
Oct 30 22:03:10.330422 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:10.330391    4473 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w2-0c99b1\"/\"openshift-service-ca.crt\""
Oct 30 22:03:10.330422 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:10.330401    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w2-0c99b1\"/\"default-dockercfg-cp7k7\""
Oct 30 22:03:10.376859 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:10.376821    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/89c09555-cf53-49a7-9c3b-95d2da2a293a-model-cache\") pod \"distinct-2-w2-0c99b1-kserve-7466864658-8z7fk\" (UID: \"89c09555-cf53-49a7-9c3b-95d2da2a293a\") " pod="e2e-models-e2e-worker-w2-0c99b1/distinct-2-w2-0c99b1-kserve-7466864658-8z7fk"
Oct 30 22:03:10.377339 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:10.377313    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/89c09555-cf53-49a7-9c3b-95d2da2a293a-dshm\") pod \"distinct-2-w2-0c99b1-kserve-7466864658-8z7fk\" (UID: \"89c09555-cf53-49a7-9c3b-95d2da2a293a\") " pod="e2e-models-e2e-worker-w2-0c99b1/distinct-2-w2-0c99b1-kserve-7466864658-8z7fk"
Oct 30 22:03:10.377710 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:10.377688    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qxqhm\" (UniqueName: \"kubernetes.io/projected/89c09555-cf53-49a7-9c3b-95d2da2a293a-kube-api-access-qxqhm\") pod \"distinct-2-w2-0c99b1-kserve-7466864658-8z7fk\" (UID: \"89c09555-cf53-49a7-9c3b-95d2da2a293a\") " pod="e2e-models-e2e-worker-w2-0c99b1/distinct-2-w2-0c99b1-kserve-7466864658-8z7fk"
Oct 30 22:03:10.378001 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:10.377977    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/89c09555-cf53-49a7-9c3b-95d2da2a293a-home\") pod \"distinct-2-w2-0c99b1-kserve-7466864658-8z7fk\" (UID: \"89c09555-cf53-49a7-9c3b-95d2da2a293a\") " pod="e2e-models-e2e-worker-w2-0c99b1/distinct-2-w2-0c99b1-kserve-7466864658-8z7fk"
Oct 30 22:03:10.378280 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:10.378215    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/89c09555-cf53-49a7-9c3b-95d2da2a293a-tls-certs\") pod \"distinct-2-w2-0c99b1-kserve-7466864658-8z7fk\" (UID: \"89c09555-cf53-49a7-9c3b-95d2da2a293a\") " pod="e2e-models-e2e-worker-w2-0c99b1/distinct-2-w2-0c99b1-kserve-7466864658-8z7fk"
Oct 30 22:03:10.405046 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:10.388086    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/89c09555-cf53-49a7-9c3b-95d2da2a293a-tmp-dir\") pod \"distinct-2-w2-0c99b1-kserve-7466864658-8z7fk\" (UID: \"89c09555-cf53-49a7-9c3b-95d2da2a293a\") " pod="e2e-models-e2e-worker-w2-0c99b1/distinct-2-w2-0c99b1-kserve-7466864658-8z7fk"
Oct 30 22:03:10.515462 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:10.515344    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/89c09555-cf53-49a7-9c3b-95d2da2a293a-home\") pod \"distinct-2-w2-0c99b1-kserve-7466864658-8z7fk\" (UID: \"89c09555-cf53-49a7-9c3b-95d2da2a293a\") " pod="e2e-models-e2e-worker-w2-0c99b1/distinct-2-w2-0c99b1-kserve-7466864658-8z7fk"
Oct 30 22:03:10.516611 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:10.516587    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/89c09555-cf53-49a7-9c3b-95d2da2a293a-tls-certs\") pod \"distinct-2-w2-0c99b1-kserve-7466864658-8z7fk\" (UID: \"89c09555-cf53-49a7-9c3b-95d2da2a293a\") " pod="e2e-models-e2e-worker-w2-0c99b1/distinct-2-w2-0c99b1-kserve-7466864658-8z7fk"
Oct 30 22:03:10.520079 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:10.519251    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/89c09555-cf53-49a7-9c3b-95d2da2a293a-tmp-dir\") pod \"distinct-2-w2-0c99b1-kserve-7466864658-8z7fk\" (UID: \"89c09555-cf53-49a7-9c3b-95d2da2a293a\") " pod="e2e-models-e2e-worker-w2-0c99b1/distinct-2-w2-0c99b1-kserve-7466864658-8z7fk"
Oct 30 22:03:10.520079 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:10.519375    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/89c09555-cf53-49a7-9c3b-95d2da2a293a-model-cache\") pod \"distinct-2-w2-0c99b1-kserve-7466864658-8z7fk\" (UID: \"89c09555-cf53-49a7-9c3b-95d2da2a293a\") " pod="e2e-models-e2e-worker-w2-0c99b1/distinct-2-w2-0c99b1-kserve-7466864658-8z7fk"
Oct 30 22:03:10.520079 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:10.519419    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/89c09555-cf53-49a7-9c3b-95d2da2a293a-dshm\") pod \"distinct-2-w2-0c99b1-kserve-7466864658-8z7fk\" (UID: \"89c09555-cf53-49a7-9c3b-95d2da2a293a\") " pod="e2e-models-e2e-worker-w2-0c99b1/distinct-2-w2-0c99b1-kserve-7466864658-8z7fk"
Oct 30 22:03:10.520079 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:10.519587    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qxqhm\" (UniqueName: \"kubernetes.io/projected/89c09555-cf53-49a7-9c3b-95d2da2a293a-kube-api-access-qxqhm\") pod \"distinct-2-w2-0c99b1-kserve-7466864658-8z7fk\" (UID: \"89c09555-cf53-49a7-9c3b-95d2da2a293a\") " pod="e2e-models-e2e-worker-w2-0c99b1/distinct-2-w2-0c99b1-kserve-7466864658-8z7fk"
Oct 30 22:03:10.520079 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:10.516431    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/89c09555-cf53-49a7-9c3b-95d2da2a293a-home\") pod \"distinct-2-w2-0c99b1-kserve-7466864658-8z7fk\" (UID: \"89c09555-cf53-49a7-9c3b-95d2da2a293a\") " pod="e2e-models-e2e-worker-w2-0c99b1/distinct-2-w2-0c99b1-kserve-7466864658-8z7fk"
Oct 30 22:03:10.520523 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:10.520339    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/89c09555-cf53-49a7-9c3b-95d2da2a293a-tmp-dir\") pod \"distinct-2-w2-0c99b1-kserve-7466864658-8z7fk\" (UID: \"89c09555-cf53-49a7-9c3b-95d2da2a293a\") " pod="e2e-models-e2e-worker-w2-0c99b1/distinct-2-w2-0c99b1-kserve-7466864658-8z7fk"
Oct 30 22:03:10.520652 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:10.520605    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/89c09555-cf53-49a7-9c3b-95d2da2a293a-model-cache\") pod \"distinct-2-w2-0c99b1-kserve-7466864658-8z7fk\" (UID: \"89c09555-cf53-49a7-9c3b-95d2da2a293a\") " pod="e2e-models-e2e-worker-w2-0c99b1/distinct-2-w2-0c99b1-kserve-7466864658-8z7fk"
Oct 30 22:03:10.524650 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:10.524582    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/89c09555-cf53-49a7-9c3b-95d2da2a293a-dshm\") pod \"distinct-2-w2-0c99b1-kserve-7466864658-8z7fk\" (UID: \"89c09555-cf53-49a7-9c3b-95d2da2a293a\") " pod="e2e-models-e2e-worker-w2-0c99b1/distinct-2-w2-0c99b1-kserve-7466864658-8z7fk"
Oct 30 22:03:10.535021 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:10.528481    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/89c09555-cf53-49a7-9c3b-95d2da2a293a-tls-certs\") pod \"distinct-2-w2-0c99b1-kserve-7466864658-8z7fk\" (UID: \"89c09555-cf53-49a7-9c3b-95d2da2a293a\") " pod="e2e-models-e2e-worker-w2-0c99b1/distinct-2-w2-0c99b1-kserve-7466864658-8z7fk"
Oct 30 22:03:10.578382 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:10.578353    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qxqhm\" (UniqueName: \"kubernetes.io/projected/89c09555-cf53-49a7-9c3b-95d2da2a293a-kube-api-access-qxqhm\") pod \"distinct-2-w2-0c99b1-kserve-7466864658-8z7fk\" (UID: \"89c09555-cf53-49a7-9c3b-95d2da2a293a\") " pod="e2e-models-e2e-worker-w2-0c99b1/distinct-2-w2-0c99b1-kserve-7466864658-8z7fk"
Oct 30 22:03:10.876059 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:10.876029    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-0c99b1/distinct-2-w2-0c99b1-kserve-7466864658-8z7fk"
Oct 30 22:03:10.888934 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:10.888888    4473 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-bc2x8"]
Oct 30 22:03:10.893646 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:10.893621    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-bc2x8"
Oct 30 22:03:10.970417 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:10.962846    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-bc2x8"]
Oct 30 22:03:11.027647 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:11.027607    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/365aaba7-1b9d-463f-9231-b0edad7e1aff-workload-socket\") pod \"e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-bc2x8\" (UID: \"365aaba7-1b9d-463f-9231-b0edad7e1aff\") " pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-bc2x8"
Oct 30 22:03:11.027872 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:11.027661    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/365aaba7-1b9d-463f-9231-b0edad7e1aff-workload-certs\") pod \"e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-bc2x8\" (UID: \"365aaba7-1b9d-463f-9231-b0edad7e1aff\") " pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-bc2x8"
Oct 30 22:03:11.027872 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:11.027680    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/365aaba7-1b9d-463f-9231-b0edad7e1aff-istio-podinfo\") pod \"e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-bc2x8\" (UID: \"365aaba7-1b9d-463f-9231-b0edad7e1aff\") " pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-bc2x8"
Oct 30 22:03:11.028037 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:11.027891    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/365aaba7-1b9d-463f-9231-b0edad7e1aff-istio-envoy\") pod \"e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-bc2x8\" (UID: \"365aaba7-1b9d-463f-9231-b0edad7e1aff\") " pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-bc2x8"
Oct 30 22:03:11.028099 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:11.028038    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/365aaba7-1b9d-463f-9231-b0edad7e1aff-credential-socket\") pod \"e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-bc2x8\" (UID: \"365aaba7-1b9d-463f-9231-b0edad7e1aff\") " pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-bc2x8"
Oct 30 22:03:11.028099 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:11.028086    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/365aaba7-1b9d-463f-9231-b0edad7e1aff-istiod-ca-cert\") pod \"e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-bc2x8\" (UID: \"365aaba7-1b9d-463f-9231-b0edad7e1aff\") " pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-bc2x8"
Oct 30 22:03:11.028234 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:11.028199    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/365aaba7-1b9d-463f-9231-b0edad7e1aff-istio-token\") pod \"e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-bc2x8\" (UID: \"365aaba7-1b9d-463f-9231-b0edad7e1aff\") " pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-bc2x8"
Oct 30 22:03:11.028344 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:11.028322    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qg69x\" (UniqueName: \"kubernetes.io/projected/365aaba7-1b9d-463f-9231-b0edad7e1aff-kube-api-access-qg69x\") pod \"e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-bc2x8\" (UID: \"365aaba7-1b9d-463f-9231-b0edad7e1aff\") " pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-bc2x8"
Oct 30 22:03:11.028424 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:11.028407    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/365aaba7-1b9d-463f-9231-b0edad7e1aff-istio-data\") pod \"e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-bc2x8\" (UID: \"365aaba7-1b9d-463f-9231-b0edad7e1aff\") " pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-bc2x8"
Oct 30 22:03:11.129530 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:11.129434    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/365aaba7-1b9d-463f-9231-b0edad7e1aff-istio-envoy\") pod \"e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-bc2x8\" (UID: \"365aaba7-1b9d-463f-9231-b0edad7e1aff\") " pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-bc2x8"
Oct 30 22:03:11.129530 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:11.129511    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/365aaba7-1b9d-463f-9231-b0edad7e1aff-credential-socket\") pod \"e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-bc2x8\" (UID: \"365aaba7-1b9d-463f-9231-b0edad7e1aff\") " pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-bc2x8"
Oct 30 22:03:11.129772 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:11.129543    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/365aaba7-1b9d-463f-9231-b0edad7e1aff-istiod-ca-cert\") pod \"e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-bc2x8\" (UID: \"365aaba7-1b9d-463f-9231-b0edad7e1aff\") " pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-bc2x8"
Oct 30 22:03:11.129772 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:11.129597    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/365aaba7-1b9d-463f-9231-b0edad7e1aff-istio-token\") pod \"e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-bc2x8\" (UID: \"365aaba7-1b9d-463f-9231-b0edad7e1aff\") " pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-bc2x8"
Oct 30 22:03:11.129772 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:11.129688    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qg69x\" (UniqueName: \"kubernetes.io/projected/365aaba7-1b9d-463f-9231-b0edad7e1aff-kube-api-access-qg69x\") pod \"e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-bc2x8\" (UID: \"365aaba7-1b9d-463f-9231-b0edad7e1aff\") " pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-bc2x8"
Oct 30 22:03:11.129772 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:11.129759    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/365aaba7-1b9d-463f-9231-b0edad7e1aff-istio-data\") pod \"e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-bc2x8\" (UID: \"365aaba7-1b9d-463f-9231-b0edad7e1aff\") " pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-bc2x8"
Oct 30 22:03:11.130022 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:11.129806    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/365aaba7-1b9d-463f-9231-b0edad7e1aff-workload-socket\") pod \"e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-bc2x8\" (UID: \"365aaba7-1b9d-463f-9231-b0edad7e1aff\") " pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-bc2x8"
Oct 30 22:03:11.130022 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:11.129861    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/365aaba7-1b9d-463f-9231-b0edad7e1aff-workload-certs\") pod \"e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-bc2x8\" (UID: \"365aaba7-1b9d-463f-9231-b0edad7e1aff\") " pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-bc2x8"
Oct 30 22:03:11.130022 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:11.129890    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/365aaba7-1b9d-463f-9231-b0edad7e1aff-istio-podinfo\") pod \"e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-bc2x8\" (UID: \"365aaba7-1b9d-463f-9231-b0edad7e1aff\") " pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-bc2x8"
Oct 30 22:03:11.130186 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:11.130060    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/365aaba7-1b9d-463f-9231-b0edad7e1aff-credential-socket\") pod \"e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-bc2x8\" (UID: \"365aaba7-1b9d-463f-9231-b0edad7e1aff\") " pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-bc2x8"
Oct 30 22:03:11.130308 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:11.130277    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/365aaba7-1b9d-463f-9231-b0edad7e1aff-istio-data\") pod \"e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-bc2x8\" (UID: \"365aaba7-1b9d-463f-9231-b0edad7e1aff\") " pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-bc2x8"
Oct 30 22:03:11.130394 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:11.130322    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/365aaba7-1b9d-463f-9231-b0edad7e1aff-workload-socket\") pod \"e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-bc2x8\" (UID: \"365aaba7-1b9d-463f-9231-b0edad7e1aff\") " pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-bc2x8"
Oct 30 22:03:11.130514 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:11.130482    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/365aaba7-1b9d-463f-9231-b0edad7e1aff-workload-certs\") pod \"e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-bc2x8\" (UID: \"365aaba7-1b9d-463f-9231-b0edad7e1aff\") " pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-bc2x8"
Oct 30 22:03:11.130755 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:11.130527    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/365aaba7-1b9d-463f-9231-b0edad7e1aff-istiod-ca-cert\") pod \"e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-bc2x8\" (UID: \"365aaba7-1b9d-463f-9231-b0edad7e1aff\") " pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-bc2x8"
Oct 30 22:03:11.132498 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:11.132473    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/365aaba7-1b9d-463f-9231-b0edad7e1aff-istio-envoy\") pod \"e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-bc2x8\" (UID: \"365aaba7-1b9d-463f-9231-b0edad7e1aff\") " pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-bc2x8"
Oct 30 22:03:11.132623 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:11.132604    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/365aaba7-1b9d-463f-9231-b0edad7e1aff-istio-podinfo\") pod \"e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-bc2x8\" (UID: \"365aaba7-1b9d-463f-9231-b0edad7e1aff\") " pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-bc2x8"
Oct 30 22:03:11.156872 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:11.156834    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w3-0f42cb/distinct-2-w3-0f42cb-kserve-fc477897b-jk8qc"
Oct 30 22:03:11.171033 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:11.171008    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w3-0f42cb/distinct-2-w3-0f42cb-kserve-fc477897b-jk8qc"
Oct 30 22:03:11.187193 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:11.187157    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qg69x\" (UniqueName: \"kubernetes.io/projected/365aaba7-1b9d-463f-9231-b0edad7e1aff-kube-api-access-qg69x\") pod \"e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-bc2x8\" (UID: \"365aaba7-1b9d-463f-9231-b0edad7e1aff\") " pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-bc2x8"
Oct 30 22:03:11.187328 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:11.187164    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/365aaba7-1b9d-463f-9231-b0edad7e1aff-istio-token\") pod \"e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-bc2x8\" (UID: \"365aaba7-1b9d-463f-9231-b0edad7e1aff\") " pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-bc2x8"
Oct 30 22:03:11.214944 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:11.214725    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-5qn5b" event={"ID":"72b2d008-08fc-41aa-9515-ad4319bee895","Type":"ContainerStarted","Data":"782e2954c480d465a8cf704a93640bc929d32ce7e16c4fe7b38a514cc033d1ae"}
Oct 30 22:03:11.214944 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:11.214788    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-5qn5b" event={"ID":"72b2d008-08fc-41aa-9515-ad4319bee895","Type":"ContainerStarted","Data":"392fd41853a153eeb0c0047fe98bb58e2b3eadacec562ed9c6aaf542c6f9efca"}
Oct 30 22:03:11.253008 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:11.252898    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-0f42cb/distinct-2-w3-0f42cb-kserve-fc477897b-jk8qc"
Oct 30 22:03:11.260138 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:11.260106    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-0c99b1/distinct-2-w2-0c99b1-kserve-7466864658-8z7fk"]
Oct 30 22:03:11.260586 ip-10-0-140-18 kubenswrapper[4473]: W1030 22:03:11.260562    4473 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod89c09555_cf53_49a7_9c3b_95d2da2a293a.slice/crio-676a7e4eb21d8f6a63c5185e7f48c98d9faa48df88307c66ad83aef25ce405bf WatchSource:0}: Error finding container 676a7e4eb21d8f6a63c5185e7f48c98d9faa48df88307c66ad83aef25ce405bf: Status 404 returned error can't find the container with id 676a7e4eb21d8f6a63c5185e7f48c98d9faa48df88307c66ad83aef25ce405bf
Oct 30 22:03:11.263892 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:11.263865    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-bc2x8"
Oct 30 22:03:11.410354 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:11.410288    4473 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-5qn5b" podStartSLOduration=2.410263709 podStartE2EDuration="2.410263709s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:03:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:03:11.408452171 +0000 UTC m=+1237.181820675" watchObservedRunningTime="2026-10-30 22:03:11.410263709 +0000 UTC m=+1237.183632200"
Oct 30 22:03:11.688951 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:11.684971    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-bc2x8"]
Oct 30 22:03:11.691097 ip-10-0-140-18 kubenswrapper[4473]: W1030 22:03:11.691050    4473 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod365aaba7_1b9d_463f_9231_b0edad7e1aff.slice/crio-410004cff69536905e4a6d5824ddaf6b8ea30988f319544f3c1f45c25e36479e WatchSource:0}: Error finding container 410004cff69536905e4a6d5824ddaf6b8ea30988f319544f3c1f45c25e36479e: Status 404 returned error can't find the container with id 410004cff69536905e4a6d5824ddaf6b8ea30988f319544f3c1f45c25e36479e
Oct 30 22:03:11.700700 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:11.700650    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:03:11.700840 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:11.700752    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:03:11.700840 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:11.700793    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:03:11.753882 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:11.753849    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-5qn5b"
Oct 30 22:03:11.755559 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:11.755534    4473 patch_prober.go:29] interesting pod/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-5qn5b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused" start-of-body=
Oct 30 22:03:11.755689 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:11.755586    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-5qn5b" podUID="72b2d008-08fc-41aa-9515-ad4319bee895" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused"
Oct 30 22:03:12.223978 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:12.223890    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-0c99b1/distinct-2-w2-0c99b1-kserve-7466864658-8z7fk" event={"ID":"89c09555-cf53-49a7-9c3b-95d2da2a293a","Type":"ContainerStarted","Data":"d9481ecfb931f5ec088cb418fe3092a6c8a8885211b0aff9d68c9157bb9e9fdc"}
Oct 30 22:03:12.223978 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:12.223981    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-0c99b1/distinct-2-w2-0c99b1-kserve-7466864658-8z7fk" event={"ID":"89c09555-cf53-49a7-9c3b-95d2da2a293a","Type":"ContainerStarted","Data":"676a7e4eb21d8f6a63c5185e7f48c98d9faa48df88307c66ad83aef25ce405bf"}
Oct 30 22:03:12.257037 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:12.253774    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-bc2x8" event={"ID":"365aaba7-1b9d-463f-9231-b0edad7e1aff","Type":"ContainerStarted","Data":"c1fa63726fee81fdaba4280e965b3a92b716cdccf4e7edacc7bdf749e71e2546"}
Oct 30 22:03:12.257037 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:12.253837    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-bc2x8" event={"ID":"365aaba7-1b9d-463f-9231-b0edad7e1aff","Type":"ContainerStarted","Data":"410004cff69536905e4a6d5824ddaf6b8ea30988f319544f3c1f45c25e36479e"}
Oct 30 22:03:12.264468 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:12.264433    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-bc2x8"
Oct 30 22:03:12.266232 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:12.266203    4473 patch_prober.go:29] interesting pod/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-bc2x8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.83:15021/healthz/ready\": dial tcp 10.133.0.83:15021: connect: connection refused" start-of-body=
Oct 30 22:03:12.266367 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:12.266263    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-bc2x8" podUID="365aaba7-1b9d-463f-9231-b0edad7e1aff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.83:15021/healthz/ready\": dial tcp 10.133.0.83:15021: connect: connection refused"
Oct 30 22:03:12.503002 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:12.498845    4473 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w2-0c99b1/distinct-2-w2-0c99b1-kserve-7466864658-8z7fk" podStartSLOduration=2.498820165 podStartE2EDuration="2.498820165s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:03:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:03:12.351276495 +0000 UTC m=+1238.124644986" watchObservedRunningTime="2026-10-30 22:03:12.498820165 +0000 UTC m=+1238.272188653"
Oct 30 22:03:12.503002 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:12.500317    4473 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-bc2x8" podStartSLOduration=2.500297614 podStartE2EDuration="2.500297614s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:03:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:03:12.482152117 +0000 UTC m=+1238.255520604" watchObservedRunningTime="2026-10-30 22:03:12.500297614 +0000 UTC m=+1238.273666102"
Oct 30 22:03:12.761005 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:12.760644    4473 patch_prober.go:29] interesting pod/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-5qn5b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused" start-of-body=
Oct 30 22:03:12.761005 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:12.760720    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-5qn5b" podUID="72b2d008-08fc-41aa-9515-ad4319bee895" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused"
Oct 30 22:03:12.879945 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:12.877966    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-0c99b1/distinct-2-w2-0c99b1-kserve-7466864658-8z7fk"
Oct 30 22:03:13.268427 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:13.268383    4473 patch_prober.go:29] interesting pod/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-bc2x8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.83:15021/healthz/ready\": dial tcp 10.133.0.83:15021: connect: connection refused" start-of-body=
Oct 30 22:03:13.268737 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:13.268702    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-bc2x8" podUID="365aaba7-1b9d-463f-9231-b0edad7e1aff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.83:15021/healthz/ready\": dial tcp 10.133.0.83:15021: connect: connection refused"
Oct 30 22:03:13.774950 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:13.774678    4473 patch_prober.go:29] interesting pod/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-5qn5b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused" start-of-body=
Oct 30 22:03:13.774950 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:13.774744    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-5qn5b" podUID="72b2d008-08fc-41aa-9515-ad4319bee895" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused"
Oct 30 22:03:14.276098 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:14.276056    4473 patch_prober.go:29] interesting pod/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-bc2x8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.83:15021/healthz/ready\": dial tcp 10.133.0.83:15021: connect: connection refused" start-of-body=
Oct 30 22:03:14.276432 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:14.276404    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-bc2x8" podUID="365aaba7-1b9d-463f-9231-b0edad7e1aff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.83:15021/healthz/ready\": dial tcp 10.133.0.83:15021: connect: connection refused"
Oct 30 22:03:14.755944 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:14.754445    4473 patch_prober.go:29] interesting pod/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-5qn5b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused" start-of-body=
Oct 30 22:03:14.755944 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:14.754531    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-5qn5b" podUID="72b2d008-08fc-41aa-9515-ad4319bee895" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused"
Oct 30 22:03:15.268948 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:15.268352    4473 patch_prober.go:29] interesting pod/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-bc2x8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.83:15021/healthz/ready\": dial tcp 10.133.0.83:15021: connect: connection refused" start-of-body=
Oct 30 22:03:15.268948 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:15.268417    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-bc2x8" podUID="365aaba7-1b9d-463f-9231-b0edad7e1aff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.83:15021/healthz/ready\": dial tcp 10.133.0.83:15021: connect: connection refused"
Oct 30 22:03:15.773946 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:15.773273    4473 patch_prober.go:29] interesting pod/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-5qn5b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused" start-of-body=
Oct 30 22:03:15.773946 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:15.773339    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-5qn5b" podUID="72b2d008-08fc-41aa-9515-ad4319bee895" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused"
Oct 30 22:03:15.981963 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:15.980967    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-dde1e8/distinct-2-w0-dde1e8-kserve-744857d57-ftt8b" podUID="577b020d-2a6b-4020-863c-cf19d4f0ca3b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.79:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:03:16.230964 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:16.223190    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-0f42cb/distinct-w3-0f42cb-kserve-fb54fdd59-n6w4l" podUID="ff3c2b25-08aa-4b70-bf57-9cf4a11d05ed" containerName="main" probeResult="failure" output="Get \"https://10.133.0.73:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:03:16.230964 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:16.223821    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-0f42cb/distinct-2-w3-0f42cb-kserve-fc477897b-jk8qc" podUID="61b49182-057e-41af-b609-1a6abf82c197" containerName="main" probeResult="failure" output="Get \"https://10.133.0.80:8000/ready\": context deadline exceeded"
Oct 30 22:03:16.266948 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:16.266339    4473 patch_prober.go:29] interesting pod/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-bc2x8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.83:15021/healthz/ready\": dial tcp 10.133.0.83:15021: connect: connection refused" start-of-body=
Oct 30 22:03:16.266948 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:16.266402    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-bc2x8" podUID="365aaba7-1b9d-463f-9231-b0edad7e1aff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.83:15021/healthz/ready\": dial tcp 10.133.0.83:15021: connect: connection refused"
Oct 30 22:03:16.695127 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:16.695057    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-9f8aea/distinct-w5-9f8aea-kserve-5b564579cd-kmxzz" podUID="b11e8b11-334f-4a9e-98c2-b0164567a8af" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:03:16.741076 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:16.741024    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-dde1e8/distinct-w0-dde1e8-kserve-75b68b4c5d-hts2k" podUID="9d72e7c9-657e-4278-8ed9-53adb75bd530" containerName="main" probeResult="failure" output="Get \"https://10.133.0.72:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:03:16.758948 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:16.758881    4473 patch_prober.go:29] interesting pod/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-5qn5b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused" start-of-body=
Oct 30 22:03:16.759236 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:16.759205    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-5qn5b" podUID="72b2d008-08fc-41aa-9515-ad4319bee895" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused"
Oct 30 22:03:17.007006 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:16.998742    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-dde1e8/distinct-2-w0-dde1e8-kserve-744857d57-ftt8b" podUID="577b020d-2a6b-4020-863c-cf19d4f0ca3b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.79:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:03:17.007006 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:16.999385    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-dde1e8/distinct-2-w0-dde1e8-kserve-744857d57-ftt8b"
Oct 30 22:03:17.356073 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:17.230900    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-0f42cb/distinct-w3-0f42cb-kserve-fb54fdd59-n6w4l" podUID="ff3c2b25-08aa-4b70-bf57-9cf4a11d05ed" containerName="main" probeResult="failure" output="Get \"https://10.133.0.73:8000/ready\": context deadline exceeded"
Oct 30 22:03:17.546517 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:17.253468    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-0f42cb/distinct-2-w3-0f42cb-kserve-fc477897b-jk8qc" podUID="61b49182-057e-41af-b609-1a6abf82c197" containerName="main" probeResult="failure" output="Get \"https://10.133.0.80:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:03:17.547451 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:17.267105    4473 patch_prober.go:29] interesting pod/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-bc2x8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.83:15021/healthz/ready\": dial tcp 10.133.0.83:15021: connect: connection refused" start-of-body=
Oct 30 22:03:17.547689 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:17.547661    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-bc2x8" podUID="365aaba7-1b9d-463f-9231-b0edad7e1aff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.83:15021/healthz/ready\": dial tcp 10.133.0.83:15021: connect: connection refused"
Oct 30 22:03:17.547807 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:17.546801    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-0f42cb/distinct-w3-0f42cb-kserve-fb54fdd59-n6w4l"
Oct 30 22:03:17.547973 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:17.547961    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-0f42cb/distinct-2-w3-0f42cb-kserve-fc477897b-jk8qc"
Oct 30 22:03:17.730810 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:17.730765    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-9f8aea/distinct-w5-9f8aea-kserve-5b564579cd-kmxzz" podUID="b11e8b11-334f-4a9e-98c2-b0164567a8af" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:03:17.743161 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:17.743130    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-9f8aea/distinct-w5-9f8aea-kserve-5b564579cd-kmxzz"
Oct 30 22:03:17.744053 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:17.744008    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-dde1e8/distinct-w0-dde1e8-kserve-75b68b4c5d-hts2k" podUID="9d72e7c9-657e-4278-8ed9-53adb75bd530" containerName="main" probeResult="failure" output="Get \"https://10.133.0.72:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:03:17.770845 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:17.744311    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-dde1e8/distinct-w0-dde1e8-kserve-75b68b4c5d-hts2k"
Oct 30 22:03:17.770845 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:17.769451    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-29cbad/premium-w6-29cbad-kserve-6c9f8d6899-qtv6c" podUID="381b11f6-3fde-4de7-9ad3-7c0891e76ba1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:03:17.852161 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:17.838075    4473 patch_prober.go:29] interesting pod/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-5qn5b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused" start-of-body=
Oct 30 22:03:17.852161 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:17.838143    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-5qn5b" podUID="72b2d008-08fc-41aa-9515-ad4319bee895" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused"
Oct 30 22:03:18.028010 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:18.001820    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-dde1e8/distinct-2-w0-dde1e8-kserve-744857d57-ftt8b" podUID="577b020d-2a6b-4020-863c-cf19d4f0ca3b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.79:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:03:18.391808 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:18.391098    4473 patch_prober.go:29] interesting pod/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-bc2x8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.83:15021/healthz/ready\": dial tcp 10.133.0.83:15021: connect: connection refused" start-of-body=
Oct 30 22:03:18.391808 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:18.391155    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-bc2x8" podUID="365aaba7-1b9d-463f-9231-b0edad7e1aff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.83:15021/healthz/ready\": dial tcp 10.133.0.83:15021: connect: connection refused"
Oct 30 22:03:18.546971 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:18.546536    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-0f42cb/distinct-w3-0f42cb-kserve-fb54fdd59-n6w4l" podUID="ff3c2b25-08aa-4b70-bf57-9cf4a11d05ed" containerName="main" probeResult="failure" output="Get \"https://10.133.0.73:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:03:18.586950 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:18.549081    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-0f42cb/distinct-2-w3-0f42cb-kserve-fc477897b-jk8qc" podUID="61b49182-057e-41af-b609-1a6abf82c197" containerName="main" probeResult="failure" output="Get \"https://10.133.0.80:8000/ready\": context deadline exceeded"
Oct 30 22:03:18.626065 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:18.589042    4473 patch_prober.go:29] interesting pod/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-vncql container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.74:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 22:03:18.626065 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:18.589097    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-vncql" podUID="06f032d9-4824-42fd-a92f-4cb00284ae32" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.74:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:03:18.735113 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:18.735068    4473 patch_prober.go:29] interesting pod/e2e-worker-w1-d516c4-openshift-default-6bd584745-xrf95 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 22:03:18.735739 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:18.735703    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-xrf95" podUID="ef9eca59-6c6f-4523-9046-4fd0611dacef" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:03:18.752708 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:18.752664    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-9f8aea/distinct-w5-9f8aea-kserve-5b564579cd-kmxzz" podUID="b11e8b11-334f-4a9e-98c2-b0164567a8af" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:03:18.753351 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:18.753321    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-dde1e8/distinct-w0-dde1e8-kserve-75b68b4c5d-hts2k" podUID="9d72e7c9-657e-4278-8ed9-53adb75bd530" containerName="main" probeResult="failure" output="Get \"https://10.133.0.72:8000/ready\": context deadline exceeded"
Oct 30 22:03:18.765945 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:18.754779    4473 patch_prober.go:29] interesting pod/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-5qn5b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused" start-of-body=
Oct 30 22:03:18.766271 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:18.766231    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-5qn5b" podUID="72b2d008-08fc-41aa-9515-ad4319bee895" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused"
Oct 30 22:03:18.823477 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:18.823113    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-29cbad/premium-w6-29cbad-kserve-6c9f8d6899-qtv6c" podUID="381b11f6-3fde-4de7-9ad3-7c0891e76ba1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.60:8000/ready\": context deadline exceeded"
Oct 30 22:03:18.858820 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:18.858783    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-29cbad/premium-w6-29cbad-kserve-6c9f8d6899-qtv6c"
Oct 30 22:03:19.125896 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:19.123778    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-dde1e8/distinct-2-w0-dde1e8-kserve-744857d57-ftt8b" podUID="577b020d-2a6b-4020-863c-cf19d4f0ca3b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.79:8000/ready\": context deadline exceeded"
Oct 30 22:03:19.332948 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:19.315618    4473 patch_prober.go:29] interesting pod/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-bc2x8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.83:15021/healthz/ready\": dial tcp 10.133.0.83:15021: connect: connection refused" start-of-body=
Oct 30 22:03:19.332948 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:19.315673    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-bc2x8" podUID="365aaba7-1b9d-463f-9231-b0edad7e1aff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.83:15021/healthz/ready\": dial tcp 10.133.0.83:15021: connect: connection refused"
Oct 30 22:03:19.387237 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:19.387037    4473 patch_prober.go:29] interesting pod/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-wrxqs container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.76:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 22:03:19.387237 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:19.387107    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-wrxqs" podUID="370d0319-cf46-40d6-8c55-e4fa23e33dd3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.76:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:03:19.750535 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:19.750424    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-0f42cb/distinct-w3-0f42cb-kserve-fb54fdd59-n6w4l" podUID="ff3c2b25-08aa-4b70-bf57-9cf4a11d05ed" containerName="main" probeResult="failure" output="Get \"https://10.133.0.73:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:03:19.786947 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:19.763277    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-0f42cb/distinct-2-w3-0f42cb-kserve-fc477897b-jk8qc" podUID="61b49182-057e-41af-b609-1a6abf82c197" containerName="main" probeResult="failure" output="Get \"https://10.133.0.80:8000/ready\": context deadline exceeded"
Oct 30 22:03:19.913954 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:19.907779    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-dde1e8/distinct-w0-dde1e8-kserve-75b68b4c5d-hts2k" podUID="9d72e7c9-657e-4278-8ed9-53adb75bd530" containerName="main" probeResult="failure" output="Get \"https://10.133.0.72:8000/ready\": context deadline exceeded"
Oct 30 22:03:19.941946 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:19.923225    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-9f8aea/distinct-w5-9f8aea-kserve-5b564579cd-kmxzz" podUID="b11e8b11-334f-4a9e-98c2-b0164567a8af" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:03:19.941946 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:19.927644    4473 patch_prober.go:29] interesting pod/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-5qn5b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused" start-of-body=
Oct 30 22:03:19.941946 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:19.927704    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-5qn5b" podUID="72b2d008-08fc-41aa-9515-ad4319bee895" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused"
Oct 30 22:03:19.993945 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:19.961236    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-29cbad/premium-w6-29cbad-kserve-6c9f8d6899-qtv6c" podUID="381b11f6-3fde-4de7-9ad3-7c0891e76ba1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:03:20.028731 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:20.028642    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-0f42cb/distinct-2-w3-0f42cb-kserve-fc477897b-jk8qc"
Oct 30 22:03:20.080261 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:20.080230    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-0f42cb/distinct-w3-0f42cb-kserve-fb54fdd59-n6w4l"
Oct 30 22:03:20.131072 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:20.131019    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-dde1e8/distinct-2-w0-dde1e8-kserve-744857d57-ftt8b" podUID="577b020d-2a6b-4020-863c-cf19d4f0ca3b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.79:8000/ready\": context deadline exceeded"
Oct 30 22:03:20.271566 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:20.271402    4473 patch_prober.go:29] interesting pod/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-bc2x8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.83:15021/healthz/ready\": dial tcp 10.133.0.83:15021: connect: connection refused" start-of-body=
Oct 30 22:03:20.272098 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:20.272064    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-bc2x8" podUID="365aaba7-1b9d-463f-9231-b0edad7e1aff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.83:15021/healthz/ready\": dial tcp 10.133.0.83:15021: connect: connection refused"
Oct 30 22:03:20.544084 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:03:20.542746    4473 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fncvg" containerName="extractor"
Oct 30 22:03:20.559620 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:20.558540    4473 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w6-29cbad/premium-w6-29cbad-kserve-6c9f8d6899-qtv6c" podUID="381b11f6-3fde-4de7-9ad3-7c0891e76ba1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.60:8000/health\": context deadline exceeded"
Oct 30 22:03:20.636112 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:20.635332    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-dde1e8/distinct-w0-dde1e8-kserve-75b68b4c5d-hts2k"
Oct 30 22:03:20.636486 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:20.636468    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-29cbad/premium-w6-29cbad-kserve-6c9f8d6899-qtv6c"
Oct 30 22:03:20.772948 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:20.770055    4473 patch_prober.go:29] interesting pod/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-5qn5b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused" start-of-body=
Oct 30 22:03:20.772948 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:20.770123    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-5qn5b" podUID="72b2d008-08fc-41aa-9515-ad4319bee895" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused"
Oct 30 22:03:20.903126 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:20.902039    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w2-0c99b1/distinct-2-w2-0c99b1-kserve-7466864658-8z7fk"
Oct 30 22:03:20.936716 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:20.935050    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-9f8aea/distinct-w5-9f8aea-kserve-5b564579cd-kmxzz" podUID="b11e8b11-334f-4a9e-98c2-b0164567a8af" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:03:21.125027 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:21.122525    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w2-0c99b1/distinct-2-w2-0c99b1-kserve-7466864658-8z7fk"
Oct 30 22:03:21.139957 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:21.135229    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-dde1e8/distinct-2-w0-dde1e8-kserve-744857d57-ftt8b" podUID="577b020d-2a6b-4020-863c-cf19d4f0ca3b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.79:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:03:21.219154 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:21.219070    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-9f8aea/distinct-w5-9f8aea-kserve-5b564579cd-kmxzz"
Oct 30 22:03:21.221028 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:21.221006    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-0c99b1/distinct-2-w2-0c99b1-kserve-7466864658-8z7fk"
Oct 30 22:03:21.237902 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:21.237863    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-dde1e8/distinct-2-w0-dde1e8-kserve-744857d57-ftt8b"
Oct 30 22:03:21.265183 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:21.265140    4473 patch_prober.go:29] interesting pod/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-bc2x8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.83:15021/healthz/ready\": dial tcp 10.133.0.83:15021: connect: connection refused" start-of-body=
Oct 30 22:03:21.265427 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:21.265406    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-bc2x8" podUID="365aaba7-1b9d-463f-9231-b0edad7e1aff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.83:15021/healthz/ready\": dial tcp 10.133.0.83:15021: connect: connection refused"
Oct 30 22:03:22.265288 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:22.265247    4473 patch_prober.go:29] interesting pod/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-bc2x8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.83:15021/healthz/ready\": dial tcp 10.133.0.83:15021: connect: connection refused" start-of-body=
Oct 30 22:03:22.265796 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:22.265321    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-bc2x8" podUID="365aaba7-1b9d-463f-9231-b0edad7e1aff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.83:15021/healthz/ready\": dial tcp 10.133.0.83:15021: connect: connection refused"
Oct 30 22:03:22.321783 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:22.321748    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-5qn5b"
Oct 30 22:03:22.322278 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:22.322248    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-5qn5b"
Oct 30 22:03:22.323190 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:22.323172    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-5qn5b"
Oct 30 22:03:23.399060 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:23.399016    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-bc2x8"
Oct 30 22:03:23.399602 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:23.399541    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-bc2x8"
Oct 30 22:03:23.400481 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:23.400462    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-bc2x8"
Oct 30 22:03:40.058249 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:40.058218    4473 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-mrlnn"]
Oct 30 22:03:40.063746 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:40.063724    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-mrlnn"
Oct 30 22:03:40.192476 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:40.192444    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/198415ba-1770-4667-8703-4693d90c8fe8-credential-socket\") pod \"e2e-worker-w6-29cbad-openshift-default-6d74d46b79-mrlnn\" (UID: \"198415ba-1770-4667-8703-4693d90c8fe8\") " pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-mrlnn"
Oct 30 22:03:40.192476 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:40.192482    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g6f74\" (UniqueName: \"kubernetes.io/projected/198415ba-1770-4667-8703-4693d90c8fe8-kube-api-access-g6f74\") pod \"e2e-worker-w6-29cbad-openshift-default-6d74d46b79-mrlnn\" (UID: \"198415ba-1770-4667-8703-4693d90c8fe8\") " pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-mrlnn"
Oct 30 22:03:40.192699 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:40.192568    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/198415ba-1770-4667-8703-4693d90c8fe8-istio-envoy\") pod \"e2e-worker-w6-29cbad-openshift-default-6d74d46b79-mrlnn\" (UID: \"198415ba-1770-4667-8703-4693d90c8fe8\") " pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-mrlnn"
Oct 30 22:03:40.192699 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:40.192681    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/198415ba-1770-4667-8703-4693d90c8fe8-istio-data\") pod \"e2e-worker-w6-29cbad-openshift-default-6d74d46b79-mrlnn\" (UID: \"198415ba-1770-4667-8703-4693d90c8fe8\") " pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-mrlnn"
Oct 30 22:03:40.192812 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:40.192790    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/198415ba-1770-4667-8703-4693d90c8fe8-istio-podinfo\") pod \"e2e-worker-w6-29cbad-openshift-default-6d74d46b79-mrlnn\" (UID: \"198415ba-1770-4667-8703-4693d90c8fe8\") " pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-mrlnn"
Oct 30 22:03:40.192855 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:40.192846    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/198415ba-1770-4667-8703-4693d90c8fe8-workload-certs\") pod \"e2e-worker-w6-29cbad-openshift-default-6d74d46b79-mrlnn\" (UID: \"198415ba-1770-4667-8703-4693d90c8fe8\") " pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-mrlnn"
Oct 30 22:03:40.192913 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:40.192902    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/198415ba-1770-4667-8703-4693d90c8fe8-istio-token\") pod \"e2e-worker-w6-29cbad-openshift-default-6d74d46b79-mrlnn\" (UID: \"198415ba-1770-4667-8703-4693d90c8fe8\") " pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-mrlnn"
Oct 30 22:03:40.193032 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:40.193011    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/198415ba-1770-4667-8703-4693d90c8fe8-istiod-ca-cert\") pod \"e2e-worker-w6-29cbad-openshift-default-6d74d46b79-mrlnn\" (UID: \"198415ba-1770-4667-8703-4693d90c8fe8\") " pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-mrlnn"
Oct 30 22:03:40.193107 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:40.193090    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/198415ba-1770-4667-8703-4693d90c8fe8-workload-socket\") pod \"e2e-worker-w6-29cbad-openshift-default-6d74d46b79-mrlnn\" (UID: \"198415ba-1770-4667-8703-4693d90c8fe8\") " pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-mrlnn"
Oct 30 22:03:40.263429 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:40.263388    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-mrlnn"]
Oct 30 22:03:40.295102 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:40.295057    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/198415ba-1770-4667-8703-4693d90c8fe8-istio-data\") pod \"e2e-worker-w6-29cbad-openshift-default-6d74d46b79-mrlnn\" (UID: \"198415ba-1770-4667-8703-4693d90c8fe8\") " pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-mrlnn"
Oct 30 22:03:40.295294 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:40.295167    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/198415ba-1770-4667-8703-4693d90c8fe8-istio-podinfo\") pod \"e2e-worker-w6-29cbad-openshift-default-6d74d46b79-mrlnn\" (UID: \"198415ba-1770-4667-8703-4693d90c8fe8\") " pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-mrlnn"
Oct 30 22:03:40.295294 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:40.295210    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/198415ba-1770-4667-8703-4693d90c8fe8-workload-certs\") pod \"e2e-worker-w6-29cbad-openshift-default-6d74d46b79-mrlnn\" (UID: \"198415ba-1770-4667-8703-4693d90c8fe8\") " pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-mrlnn"
Oct 30 22:03:40.295294 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:40.295269    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/198415ba-1770-4667-8703-4693d90c8fe8-istio-token\") pod \"e2e-worker-w6-29cbad-openshift-default-6d74d46b79-mrlnn\" (UID: \"198415ba-1770-4667-8703-4693d90c8fe8\") " pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-mrlnn"
Oct 30 22:03:40.295478 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:40.295355    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/198415ba-1770-4667-8703-4693d90c8fe8-istiod-ca-cert\") pod \"e2e-worker-w6-29cbad-openshift-default-6d74d46b79-mrlnn\" (UID: \"198415ba-1770-4667-8703-4693d90c8fe8\") " pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-mrlnn"
Oct 30 22:03:40.295478 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:40.295449    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/198415ba-1770-4667-8703-4693d90c8fe8-workload-socket\") pod \"e2e-worker-w6-29cbad-openshift-default-6d74d46b79-mrlnn\" (UID: \"198415ba-1770-4667-8703-4693d90c8fe8\") " pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-mrlnn"
Oct 30 22:03:40.295478 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:40.295474    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/198415ba-1770-4667-8703-4693d90c8fe8-credential-socket\") pod \"e2e-worker-w6-29cbad-openshift-default-6d74d46b79-mrlnn\" (UID: \"198415ba-1770-4667-8703-4693d90c8fe8\") " pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-mrlnn"
Oct 30 22:03:40.295630 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:40.295501    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g6f74\" (UniqueName: \"kubernetes.io/projected/198415ba-1770-4667-8703-4693d90c8fe8-kube-api-access-g6f74\") pod \"e2e-worker-w6-29cbad-openshift-default-6d74d46b79-mrlnn\" (UID: \"198415ba-1770-4667-8703-4693d90c8fe8\") " pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-mrlnn"
Oct 30 22:03:40.295630 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:40.295547    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/198415ba-1770-4667-8703-4693d90c8fe8-istio-data\") pod \"e2e-worker-w6-29cbad-openshift-default-6d74d46b79-mrlnn\" (UID: \"198415ba-1770-4667-8703-4693d90c8fe8\") " pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-mrlnn"
Oct 30 22:03:40.295630 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:40.295562    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/198415ba-1770-4667-8703-4693d90c8fe8-istio-envoy\") pod \"e2e-worker-w6-29cbad-openshift-default-6d74d46b79-mrlnn\" (UID: \"198415ba-1770-4667-8703-4693d90c8fe8\") " pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-mrlnn"
Oct 30 22:03:40.296315 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:40.296290    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/198415ba-1770-4667-8703-4693d90c8fe8-workload-certs\") pod \"e2e-worker-w6-29cbad-openshift-default-6d74d46b79-mrlnn\" (UID: \"198415ba-1770-4667-8703-4693d90c8fe8\") " pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-mrlnn"
Oct 30 22:03:40.296480 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:40.296390    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/198415ba-1770-4667-8703-4693d90c8fe8-istiod-ca-cert\") pod \"e2e-worker-w6-29cbad-openshift-default-6d74d46b79-mrlnn\" (UID: \"198415ba-1770-4667-8703-4693d90c8fe8\") " pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-mrlnn"
Oct 30 22:03:40.296582 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:40.296418    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/198415ba-1770-4667-8703-4693d90c8fe8-credential-socket\") pod \"e2e-worker-w6-29cbad-openshift-default-6d74d46b79-mrlnn\" (UID: \"198415ba-1770-4667-8703-4693d90c8fe8\") " pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-mrlnn"
Oct 30 22:03:40.296681 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:40.296647    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/198415ba-1770-4667-8703-4693d90c8fe8-workload-socket\") pod \"e2e-worker-w6-29cbad-openshift-default-6d74d46b79-mrlnn\" (UID: \"198415ba-1770-4667-8703-4693d90c8fe8\") " pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-mrlnn"
Oct 30 22:03:40.299189 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:40.299161    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/198415ba-1770-4667-8703-4693d90c8fe8-istio-envoy\") pod \"e2e-worker-w6-29cbad-openshift-default-6d74d46b79-mrlnn\" (UID: \"198415ba-1770-4667-8703-4693d90c8fe8\") " pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-mrlnn"
Oct 30 22:03:40.299644 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:40.299625    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/198415ba-1770-4667-8703-4693d90c8fe8-istio-podinfo\") pod \"e2e-worker-w6-29cbad-openshift-default-6d74d46b79-mrlnn\" (UID: \"198415ba-1770-4667-8703-4693d90c8fe8\") " pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-mrlnn"
Oct 30 22:03:40.355613 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:40.355543    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/198415ba-1770-4667-8703-4693d90c8fe8-istio-token\") pod \"e2e-worker-w6-29cbad-openshift-default-6d74d46b79-mrlnn\" (UID: \"198415ba-1770-4667-8703-4693d90c8fe8\") " pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-mrlnn"
Oct 30 22:03:40.355814 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:40.355777    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g6f74\" (UniqueName: \"kubernetes.io/projected/198415ba-1770-4667-8703-4693d90c8fe8-kube-api-access-g6f74\") pod \"e2e-worker-w6-29cbad-openshift-default-6d74d46b79-mrlnn\" (UID: \"198415ba-1770-4667-8703-4693d90c8fe8\") " pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-mrlnn"
Oct 30 22:03:40.378307 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:40.378268    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-mrlnn"
Oct 30 22:03:40.412124 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:40.412092    4473 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-6r982"]
Oct 30 22:03:40.416756 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:40.416729    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-6r982"
Oct 30 22:03:40.482965 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:40.481862    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-6r982"]
Oct 30 22:03:40.500306 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:40.499485    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/48aa5363-fb6c-4cfe-a76d-e8aec041e46f-credential-socket\") pod \"e2e-worker-w1-d516c4-openshift-default-6bd584745-6r982\" (UID: \"48aa5363-fb6c-4cfe-a76d-e8aec041e46f\") " pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-6r982"
Oct 30 22:03:40.500306 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:40.499583    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/48aa5363-fb6c-4cfe-a76d-e8aec041e46f-istio-token\") pod \"e2e-worker-w1-d516c4-openshift-default-6bd584745-6r982\" (UID: \"48aa5363-fb6c-4cfe-a76d-e8aec041e46f\") " pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-6r982"
Oct 30 22:03:40.500306 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:40.499610    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/48aa5363-fb6c-4cfe-a76d-e8aec041e46f-istiod-ca-cert\") pod \"e2e-worker-w1-d516c4-openshift-default-6bd584745-6r982\" (UID: \"48aa5363-fb6c-4cfe-a76d-e8aec041e46f\") " pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-6r982"
Oct 30 22:03:40.500306 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:40.499676    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/48aa5363-fb6c-4cfe-a76d-e8aec041e46f-workload-socket\") pod \"e2e-worker-w1-d516c4-openshift-default-6bd584745-6r982\" (UID: \"48aa5363-fb6c-4cfe-a76d-e8aec041e46f\") " pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-6r982"
Oct 30 22:03:40.500306 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:40.499704    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kh2bx\" (UniqueName: \"kubernetes.io/projected/48aa5363-fb6c-4cfe-a76d-e8aec041e46f-kube-api-access-kh2bx\") pod \"e2e-worker-w1-d516c4-openshift-default-6bd584745-6r982\" (UID: \"48aa5363-fb6c-4cfe-a76d-e8aec041e46f\") " pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-6r982"
Oct 30 22:03:40.500306 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:40.499724    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/48aa5363-fb6c-4cfe-a76d-e8aec041e46f-workload-certs\") pod \"e2e-worker-w1-d516c4-openshift-default-6bd584745-6r982\" (UID: \"48aa5363-fb6c-4cfe-a76d-e8aec041e46f\") " pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-6r982"
Oct 30 22:03:40.500306 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:40.499739    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/48aa5363-fb6c-4cfe-a76d-e8aec041e46f-istio-podinfo\") pod \"e2e-worker-w1-d516c4-openshift-default-6bd584745-6r982\" (UID: \"48aa5363-fb6c-4cfe-a76d-e8aec041e46f\") " pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-6r982"
Oct 30 22:03:40.500306 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:40.499820    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/48aa5363-fb6c-4cfe-a76d-e8aec041e46f-istio-data\") pod \"e2e-worker-w1-d516c4-openshift-default-6bd584745-6r982\" (UID: \"48aa5363-fb6c-4cfe-a76d-e8aec041e46f\") " pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-6r982"
Oct 30 22:03:40.500306 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:40.499843    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/48aa5363-fb6c-4cfe-a76d-e8aec041e46f-istio-envoy\") pod \"e2e-worker-w1-d516c4-openshift-default-6bd584745-6r982\" (UID: \"48aa5363-fb6c-4cfe-a76d-e8aec041e46f\") " pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-6r982"
Oct 30 22:03:40.600816 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:40.600777    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/48aa5363-fb6c-4cfe-a76d-e8aec041e46f-istio-token\") pod \"e2e-worker-w1-d516c4-openshift-default-6bd584745-6r982\" (UID: \"48aa5363-fb6c-4cfe-a76d-e8aec041e46f\") " pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-6r982"
Oct 30 22:03:40.601040 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:40.600835    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/48aa5363-fb6c-4cfe-a76d-e8aec041e46f-istiod-ca-cert\") pod \"e2e-worker-w1-d516c4-openshift-default-6bd584745-6r982\" (UID: \"48aa5363-fb6c-4cfe-a76d-e8aec041e46f\") " pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-6r982"
Oct 30 22:03:40.601040 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:40.600954    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/48aa5363-fb6c-4cfe-a76d-e8aec041e46f-workload-socket\") pod \"e2e-worker-w1-d516c4-openshift-default-6bd584745-6r982\" (UID: \"48aa5363-fb6c-4cfe-a76d-e8aec041e46f\") " pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-6r982"
Oct 30 22:03:40.601169 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:40.601088    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kh2bx\" (UniqueName: \"kubernetes.io/projected/48aa5363-fb6c-4cfe-a76d-e8aec041e46f-kube-api-access-kh2bx\") pod \"e2e-worker-w1-d516c4-openshift-default-6bd584745-6r982\" (UID: \"48aa5363-fb6c-4cfe-a76d-e8aec041e46f\") " pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-6r982"
Oct 30 22:03:40.601169 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:40.601145    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/48aa5363-fb6c-4cfe-a76d-e8aec041e46f-workload-certs\") pod \"e2e-worker-w1-d516c4-openshift-default-6bd584745-6r982\" (UID: \"48aa5363-fb6c-4cfe-a76d-e8aec041e46f\") " pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-6r982"
Oct 30 22:03:40.601275 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:40.601172    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/48aa5363-fb6c-4cfe-a76d-e8aec041e46f-istio-podinfo\") pod \"e2e-worker-w1-d516c4-openshift-default-6bd584745-6r982\" (UID: \"48aa5363-fb6c-4cfe-a76d-e8aec041e46f\") " pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-6r982"
Oct 30 22:03:40.601325 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:40.601281    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/48aa5363-fb6c-4cfe-a76d-e8aec041e46f-istio-data\") pod \"e2e-worker-w1-d516c4-openshift-default-6bd584745-6r982\" (UID: \"48aa5363-fb6c-4cfe-a76d-e8aec041e46f\") " pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-6r982"
Oct 30 22:03:40.601325 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:40.601318    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/48aa5363-fb6c-4cfe-a76d-e8aec041e46f-istio-envoy\") pod \"e2e-worker-w1-d516c4-openshift-default-6bd584745-6r982\" (UID: \"48aa5363-fb6c-4cfe-a76d-e8aec041e46f\") " pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-6r982"
Oct 30 22:03:40.601420 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:40.601382    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/48aa5363-fb6c-4cfe-a76d-e8aec041e46f-workload-socket\") pod \"e2e-worker-w1-d516c4-openshift-default-6bd584745-6r982\" (UID: \"48aa5363-fb6c-4cfe-a76d-e8aec041e46f\") " pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-6r982"
Oct 30 22:03:40.601479 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:40.601451    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/48aa5363-fb6c-4cfe-a76d-e8aec041e46f-workload-certs\") pod \"e2e-worker-w1-d516c4-openshift-default-6bd584745-6r982\" (UID: \"48aa5363-fb6c-4cfe-a76d-e8aec041e46f\") " pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-6r982"
Oct 30 22:03:40.601479 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:40.601460    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/48aa5363-fb6c-4cfe-a76d-e8aec041e46f-credential-socket\") pod \"e2e-worker-w1-d516c4-openshift-default-6bd584745-6r982\" (UID: \"48aa5363-fb6c-4cfe-a76d-e8aec041e46f\") " pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-6r982"
Oct 30 22:03:40.601774 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:40.601752    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/48aa5363-fb6c-4cfe-a76d-e8aec041e46f-istiod-ca-cert\") pod \"e2e-worker-w1-d516c4-openshift-default-6bd584745-6r982\" (UID: \"48aa5363-fb6c-4cfe-a76d-e8aec041e46f\") " pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-6r982"
Oct 30 22:03:40.601882 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:40.601780    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/48aa5363-fb6c-4cfe-a76d-e8aec041e46f-istio-data\") pod \"e2e-worker-w1-d516c4-openshift-default-6bd584745-6r982\" (UID: \"48aa5363-fb6c-4cfe-a76d-e8aec041e46f\") " pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-6r982"
Oct 30 22:03:40.601882 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:40.601809    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/48aa5363-fb6c-4cfe-a76d-e8aec041e46f-credential-socket\") pod \"e2e-worker-w1-d516c4-openshift-default-6bd584745-6r982\" (UID: \"48aa5363-fb6c-4cfe-a76d-e8aec041e46f\") " pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-6r982"
Oct 30 22:03:40.603878 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:40.603855    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/48aa5363-fb6c-4cfe-a76d-e8aec041e46f-istio-envoy\") pod \"e2e-worker-w1-d516c4-openshift-default-6bd584745-6r982\" (UID: \"48aa5363-fb6c-4cfe-a76d-e8aec041e46f\") " pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-6r982"
Oct 30 22:03:40.604034 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:40.604016    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/48aa5363-fb6c-4cfe-a76d-e8aec041e46f-istio-podinfo\") pod \"e2e-worker-w1-d516c4-openshift-default-6bd584745-6r982\" (UID: \"48aa5363-fb6c-4cfe-a76d-e8aec041e46f\") " pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-6r982"
Oct 30 22:03:40.659164 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:40.659135    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/48aa5363-fb6c-4cfe-a76d-e8aec041e46f-istio-token\") pod \"e2e-worker-w1-d516c4-openshift-default-6bd584745-6r982\" (UID: \"48aa5363-fb6c-4cfe-a76d-e8aec041e46f\") " pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-6r982"
Oct 30 22:03:40.659350 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:40.659270    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kh2bx\" (UniqueName: \"kubernetes.io/projected/48aa5363-fb6c-4cfe-a76d-e8aec041e46f-kube-api-access-kh2bx\") pod \"e2e-worker-w1-d516c4-openshift-default-6bd584745-6r982\" (UID: \"48aa5363-fb6c-4cfe-a76d-e8aec041e46f\") " pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-6r982"
Oct 30 22:03:40.735209 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:40.735155    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-6r982"
Oct 30 22:03:40.820876 ip-10-0-140-18 kubenswrapper[4473]: W1030 22:03:40.820833    4473 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod198415ba_1770_4667_8703_4693d90c8fe8.slice/crio-620f1e53e16b54c81ffdb4e141d0d665a9fa06c5052b76ec62ae6c1c58372152 WatchSource:0}: Error finding container 620f1e53e16b54c81ffdb4e141d0d665a9fa06c5052b76ec62ae6c1c58372152: Status 404 returned error can't find the container with id 620f1e53e16b54c81ffdb4e141d0d665a9fa06c5052b76ec62ae6c1c58372152
Oct 30 22:03:40.822780 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:40.822674    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-mrlnn"]
Oct 30 22:03:40.824985 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:40.824943    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:03:40.825192 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:40.825168    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:03:40.825341 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:40.825321    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:03:41.135951 ip-10-0-140-18 kubenswrapper[4473]: W1030 22:03:41.135901    4473 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod48aa5363_fb6c_4cfe_a76d_e8aec041e46f.slice/crio-ab7528d99c6a2572c34322a6ec4de3f95fbcec5e1e9c51250b5cd49d1792ccb5 WatchSource:0}: Error finding container ab7528d99c6a2572c34322a6ec4de3f95fbcec5e1e9c51250b5cd49d1792ccb5: Status 404 returned error can't find the container with id ab7528d99c6a2572c34322a6ec4de3f95fbcec5e1e9c51250b5cd49d1792ccb5
Oct 30 22:03:41.136364 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:41.136001    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-6r982"]
Oct 30 22:03:41.138851 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:41.138805    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:03:41.139021 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:41.138896    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:03:41.139021 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:41.138956    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:03:41.355948 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:41.355713    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-mrlnn" event={"ID":"198415ba-1770-4667-8703-4693d90c8fe8","Type":"ContainerStarted","Data":"0516329edc3602ca725ecb3061ec2b3bc02f8407fedd9323c226f0393d0b800f"}
Oct 30 22:03:41.355948 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:41.355793    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-mrlnn" event={"ID":"198415ba-1770-4667-8703-4693d90c8fe8","Type":"ContainerStarted","Data":"620f1e53e16b54c81ffdb4e141d0d665a9fa06c5052b76ec62ae6c1c58372152"}
Oct 30 22:03:41.384672 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:41.384619    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-mrlnn"
Oct 30 22:03:41.392614 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:41.391497    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-6r982" event={"ID":"48aa5363-fb6c-4cfe-a76d-e8aec041e46f","Type":"ContainerStarted","Data":"e96649e74ccb53a7dd66222b8dc1373b948644c41d56f4fae45a6d98e6a082d3"}
Oct 30 22:03:41.392614 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:41.391564    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-6r982" event={"ID":"48aa5363-fb6c-4cfe-a76d-e8aec041e46f","Type":"ContainerStarted","Data":"ab7528d99c6a2572c34322a6ec4de3f95fbcec5e1e9c51250b5cd49d1792ccb5"}
Oct 30 22:03:41.393074 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:41.393049    4473 patch_prober.go:29] interesting pod/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-mrlnn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.84:15021/healthz/ready\": dial tcp 10.133.0.84:15021: connect: connection refused" start-of-body=
Oct 30 22:03:41.393365 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:41.393336    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-mrlnn" podUID="198415ba-1770-4667-8703-4693d90c8fe8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.84:15021/healthz/ready\": dial tcp 10.133.0.84:15021: connect: connection refused"
Oct 30 22:03:41.498613 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:41.498498    4473 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-mrlnn" podStartSLOduration=1.498481753 podStartE2EDuration="1.498481753s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:03:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:03:41.496469222 +0000 UTC m=+1267.269837709" watchObservedRunningTime="2026-10-30 22:03:41.498481753 +0000 UTC m=+1267.271850240"
Oct 30 22:03:41.618546 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:41.618495    4473 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-6r982" podStartSLOduration=1.618481644 podStartE2EDuration="1.618481644s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:03:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:03:41.616676037 +0000 UTC m=+1267.390044524" watchObservedRunningTime="2026-10-30 22:03:41.618481644 +0000 UTC m=+1267.391850133"
Oct 30 22:03:41.735637 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:41.735604    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-6r982"
Oct 30 22:03:42.379946 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:42.379669    4473 patch_prober.go:29] interesting pod/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-mrlnn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.84:15021/healthz/ready\": dial tcp 10.133.0.84:15021: connect: connection refused" start-of-body=
Oct 30 22:03:42.379946 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:42.379750    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-mrlnn" podUID="198415ba-1770-4667-8703-4693d90c8fe8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.84:15021/healthz/ready\": dial tcp 10.133.0.84:15021: connect: connection refused"
Oct 30 22:03:42.753003 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:42.752146    4473 patch_prober.go:29] interesting pod/e2e-worker-w1-d516c4-openshift-default-6bd584745-6r982 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused" start-of-body=
Oct 30 22:03:42.753003 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:42.752243    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-6r982" podUID="48aa5363-fb6c-4cfe-a76d-e8aec041e46f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused"
Oct 30 22:03:43.379540 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:43.379506    4473 patch_prober.go:29] interesting pod/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-mrlnn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.84:15021/healthz/ready\": dial tcp 10.133.0.84:15021: connect: connection refused" start-of-body=
Oct 30 22:03:43.379721 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:43.379557    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-mrlnn" podUID="198415ba-1770-4667-8703-4693d90c8fe8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.84:15021/healthz/ready\": dial tcp 10.133.0.84:15021: connect: connection refused"
Oct 30 22:03:43.751671 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:43.751549    4473 patch_prober.go:29] interesting pod/e2e-worker-w1-d516c4-openshift-default-6bd584745-6r982 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused" start-of-body=
Oct 30 22:03:43.752398 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:43.752365    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-6r982" podUID="48aa5363-fb6c-4cfe-a76d-e8aec041e46f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused"
Oct 30 22:03:44.390002 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:44.389953    4473 patch_prober.go:29] interesting pod/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-mrlnn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.84:15021/healthz/ready\": dial tcp 10.133.0.84:15021: connect: connection refused" start-of-body=
Oct 30 22:03:44.390342 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:44.390310    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-mrlnn" podUID="198415ba-1770-4667-8703-4693d90c8fe8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.84:15021/healthz/ready\": dial tcp 10.133.0.84:15021: connect: connection refused"
Oct 30 22:03:44.746607 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:44.746509    4473 patch_prober.go:29] interesting pod/e2e-worker-w1-d516c4-openshift-default-6bd584745-6r982 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused" start-of-body=
Oct 30 22:03:44.746899 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:44.746868    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-6r982" podUID="48aa5363-fb6c-4cfe-a76d-e8aec041e46f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused"
Oct 30 22:03:45.380437 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:45.380306    4473 patch_prober.go:29] interesting pod/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-mrlnn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.84:15021/healthz/ready\": dial tcp 10.133.0.84:15021: connect: connection refused" start-of-body=
Oct 30 22:03:45.380437 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:45.380401    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-mrlnn" podUID="198415ba-1770-4667-8703-4693d90c8fe8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.84:15021/healthz/ready\": dial tcp 10.133.0.84:15021: connect: connection refused"
Oct 30 22:03:45.743909 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:45.743367    4473 patch_prober.go:29] interesting pod/e2e-worker-w1-d516c4-openshift-default-6bd584745-6r982 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused" start-of-body=
Oct 30 22:03:45.743909 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:45.743437    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-6r982" podUID="48aa5363-fb6c-4cfe-a76d-e8aec041e46f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused"
Oct 30 22:03:46.382971 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:46.381391    4473 patch_prober.go:29] interesting pod/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-mrlnn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.84:15021/healthz/ready\": dial tcp 10.133.0.84:15021: connect: connection refused" start-of-body=
Oct 30 22:03:46.382971 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:46.381449    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-mrlnn" podUID="198415ba-1770-4667-8703-4693d90c8fe8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.84:15021/healthz/ready\": dial tcp 10.133.0.84:15021: connect: connection refused"
Oct 30 22:03:46.749388 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:46.749285    4473 patch_prober.go:29] interesting pod/e2e-worker-w1-d516c4-openshift-default-6bd584745-6r982 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused" start-of-body=
Oct 30 22:03:46.749388 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:46.749351    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-6r982" podUID="48aa5363-fb6c-4cfe-a76d-e8aec041e46f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused"
Oct 30 22:03:47.381066 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:47.378727    4473 patch_prober.go:29] interesting pod/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-mrlnn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.84:15021/healthz/ready\": dial tcp 10.133.0.84:15021: connect: connection refused" start-of-body=
Oct 30 22:03:47.381066 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:47.378807    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-mrlnn" podUID="198415ba-1770-4667-8703-4693d90c8fe8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.84:15021/healthz/ready\": dial tcp 10.133.0.84:15021: connect: connection refused"
Oct 30 22:03:47.745000 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:47.736360    4473 patch_prober.go:29] interesting pod/e2e-worker-w1-d516c4-openshift-default-6bd584745-6r982 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused" start-of-body=
Oct 30 22:03:47.745000 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:47.736428    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-6r982" podUID="48aa5363-fb6c-4cfe-a76d-e8aec041e46f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused"
Oct 30 22:03:48.379298 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:48.379253    4473 patch_prober.go:29] interesting pod/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-mrlnn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.84:15021/healthz/ready\": dial tcp 10.133.0.84:15021: connect: connection refused" start-of-body=
Oct 30 22:03:48.379484 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:48.379346    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-mrlnn" podUID="198415ba-1770-4667-8703-4693d90c8fe8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.84:15021/healthz/ready\": dial tcp 10.133.0.84:15021: connect: connection refused"
Oct 30 22:03:49.254023 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:49.252771    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-6r982"
Oct 30 22:03:49.254532 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:49.254439    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-6r982"
Oct 30 22:03:49.267247 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:49.267210    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-6r982"
Oct 30 22:03:49.517997 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:49.511659    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-mrlnn"
Oct 30 22:03:49.532480 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:49.532386    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-mrlnn"
Oct 30 22:03:49.604845 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:03:49.604733    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-mrlnn"
Oct 30 22:03:50.362044 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:03:50.361996    4473 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fncvg" containerName="extractor"
Oct 30 22:04:09.325531 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:09.325496    4473 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-ktzwz"]
Oct 30 22:04:09.329968 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:09.329905    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-ktzwz"
Oct 30 22:04:09.440120 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:09.440070    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-ktzwz"]
Oct 30 22:04:09.521371 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:09.521330    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/df9bbdb1-5a45-4223-b03a-5d5e11542e60-workload-socket\") pod \"e2e-worker-w2-0c99b1-openshift-default-967855977-ktzwz\" (UID: \"df9bbdb1-5a45-4223-b03a-5d5e11542e60\") " pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-ktzwz"
Oct 30 22:04:09.521584 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:09.521395    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/df9bbdb1-5a45-4223-b03a-5d5e11542e60-workload-certs\") pod \"e2e-worker-w2-0c99b1-openshift-default-967855977-ktzwz\" (UID: \"df9bbdb1-5a45-4223-b03a-5d5e11542e60\") " pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-ktzwz"
Oct 30 22:04:09.521724 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:09.521700    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/df9bbdb1-5a45-4223-b03a-5d5e11542e60-istio-data\") pod \"e2e-worker-w2-0c99b1-openshift-default-967855977-ktzwz\" (UID: \"df9bbdb1-5a45-4223-b03a-5d5e11542e60\") " pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-ktzwz"
Oct 30 22:04:09.521791 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:09.521731    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/df9bbdb1-5a45-4223-b03a-5d5e11542e60-istiod-ca-cert\") pod \"e2e-worker-w2-0c99b1-openshift-default-967855977-ktzwz\" (UID: \"df9bbdb1-5a45-4223-b03a-5d5e11542e60\") " pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-ktzwz"
Oct 30 22:04:09.521865 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:09.521841    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/df9bbdb1-5a45-4223-b03a-5d5e11542e60-istio-podinfo\") pod \"e2e-worker-w2-0c99b1-openshift-default-967855977-ktzwz\" (UID: \"df9bbdb1-5a45-4223-b03a-5d5e11542e60\") " pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-ktzwz"
Oct 30 22:04:09.522039 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:09.522012    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/df9bbdb1-5a45-4223-b03a-5d5e11542e60-istio-envoy\") pod \"e2e-worker-w2-0c99b1-openshift-default-967855977-ktzwz\" (UID: \"df9bbdb1-5a45-4223-b03a-5d5e11542e60\") " pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-ktzwz"
Oct 30 22:04:09.522115 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:09.522089    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t9px6\" (UniqueName: \"kubernetes.io/projected/df9bbdb1-5a45-4223-b03a-5d5e11542e60-kube-api-access-t9px6\") pod \"e2e-worker-w2-0c99b1-openshift-default-967855977-ktzwz\" (UID: \"df9bbdb1-5a45-4223-b03a-5d5e11542e60\") " pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-ktzwz"
Oct 30 22:04:09.522192 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:09.522170    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/df9bbdb1-5a45-4223-b03a-5d5e11542e60-istio-token\") pod \"e2e-worker-w2-0c99b1-openshift-default-967855977-ktzwz\" (UID: \"df9bbdb1-5a45-4223-b03a-5d5e11542e60\") " pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-ktzwz"
Oct 30 22:04:09.522256 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:09.522217    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/df9bbdb1-5a45-4223-b03a-5d5e11542e60-credential-socket\") pod \"e2e-worker-w2-0c99b1-openshift-default-967855977-ktzwz\" (UID: \"df9bbdb1-5a45-4223-b03a-5d5e11542e60\") " pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-ktzwz"
Oct 30 22:04:09.623284 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:09.623169    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t9px6\" (UniqueName: \"kubernetes.io/projected/df9bbdb1-5a45-4223-b03a-5d5e11542e60-kube-api-access-t9px6\") pod \"e2e-worker-w2-0c99b1-openshift-default-967855977-ktzwz\" (UID: \"df9bbdb1-5a45-4223-b03a-5d5e11542e60\") " pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-ktzwz"
Oct 30 22:04:09.623284 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:09.623221    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/df9bbdb1-5a45-4223-b03a-5d5e11542e60-istio-token\") pod \"e2e-worker-w2-0c99b1-openshift-default-967855977-ktzwz\" (UID: \"df9bbdb1-5a45-4223-b03a-5d5e11542e60\") " pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-ktzwz"
Oct 30 22:04:09.623284 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:09.623259    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/df9bbdb1-5a45-4223-b03a-5d5e11542e60-credential-socket\") pod \"e2e-worker-w2-0c99b1-openshift-default-967855977-ktzwz\" (UID: \"df9bbdb1-5a45-4223-b03a-5d5e11542e60\") " pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-ktzwz"
Oct 30 22:04:09.623801 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:09.623765    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/df9bbdb1-5a45-4223-b03a-5d5e11542e60-workload-socket\") pod \"e2e-worker-w2-0c99b1-openshift-default-967855977-ktzwz\" (UID: \"df9bbdb1-5a45-4223-b03a-5d5e11542e60\") " pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-ktzwz"
Oct 30 22:04:09.623949 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:09.623831    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/df9bbdb1-5a45-4223-b03a-5d5e11542e60-credential-socket\") pod \"e2e-worker-w2-0c99b1-openshift-default-967855977-ktzwz\" (UID: \"df9bbdb1-5a45-4223-b03a-5d5e11542e60\") " pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-ktzwz"
Oct 30 22:04:09.623949 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:09.623854    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/df9bbdb1-5a45-4223-b03a-5d5e11542e60-workload-certs\") pod \"e2e-worker-w2-0c99b1-openshift-default-967855977-ktzwz\" (UID: \"df9bbdb1-5a45-4223-b03a-5d5e11542e60\") " pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-ktzwz"
Oct 30 22:04:09.624050 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:09.623979    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/df9bbdb1-5a45-4223-b03a-5d5e11542e60-workload-socket\") pod \"e2e-worker-w2-0c99b1-openshift-default-967855977-ktzwz\" (UID: \"df9bbdb1-5a45-4223-b03a-5d5e11542e60\") " pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-ktzwz"
Oct 30 22:04:09.624117 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:09.624099    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/df9bbdb1-5a45-4223-b03a-5d5e11542e60-istio-data\") pod \"e2e-worker-w2-0c99b1-openshift-default-967855977-ktzwz\" (UID: \"df9bbdb1-5a45-4223-b03a-5d5e11542e60\") " pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-ktzwz"
Oct 30 22:04:09.624152 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:09.624115    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/df9bbdb1-5a45-4223-b03a-5d5e11542e60-workload-certs\") pod \"e2e-worker-w2-0c99b1-openshift-default-967855977-ktzwz\" (UID: \"df9bbdb1-5a45-4223-b03a-5d5e11542e60\") " pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-ktzwz"
Oct 30 22:04:09.624152 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:09.624131    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/df9bbdb1-5a45-4223-b03a-5d5e11542e60-istiod-ca-cert\") pod \"e2e-worker-w2-0c99b1-openshift-default-967855977-ktzwz\" (UID: \"df9bbdb1-5a45-4223-b03a-5d5e11542e60\") " pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-ktzwz"
Oct 30 22:04:09.624215 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:09.624185    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/df9bbdb1-5a45-4223-b03a-5d5e11542e60-istio-podinfo\") pod \"e2e-worker-w2-0c99b1-openshift-default-967855977-ktzwz\" (UID: \"df9bbdb1-5a45-4223-b03a-5d5e11542e60\") " pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-ktzwz"
Oct 30 22:04:09.624255 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:09.624241    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/df9bbdb1-5a45-4223-b03a-5d5e11542e60-istio-envoy\") pod \"e2e-worker-w2-0c99b1-openshift-default-967855977-ktzwz\" (UID: \"df9bbdb1-5a45-4223-b03a-5d5e11542e60\") " pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-ktzwz"
Oct 30 22:04:09.624373 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:09.624354    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/df9bbdb1-5a45-4223-b03a-5d5e11542e60-istio-data\") pod \"e2e-worker-w2-0c99b1-openshift-default-967855977-ktzwz\" (UID: \"df9bbdb1-5a45-4223-b03a-5d5e11542e60\") " pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-ktzwz"
Oct 30 22:04:09.624687 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:09.624669    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/df9bbdb1-5a45-4223-b03a-5d5e11542e60-istiod-ca-cert\") pod \"e2e-worker-w2-0c99b1-openshift-default-967855977-ktzwz\" (UID: \"df9bbdb1-5a45-4223-b03a-5d5e11542e60\") " pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-ktzwz"
Oct 30 22:04:09.626764 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:09.626742    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/df9bbdb1-5a45-4223-b03a-5d5e11542e60-istio-envoy\") pod \"e2e-worker-w2-0c99b1-openshift-default-967855977-ktzwz\" (UID: \"df9bbdb1-5a45-4223-b03a-5d5e11542e60\") " pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-ktzwz"
Oct 30 22:04:09.626912 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:09.626893    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/df9bbdb1-5a45-4223-b03a-5d5e11542e60-istio-podinfo\") pod \"e2e-worker-w2-0c99b1-openshift-default-967855977-ktzwz\" (UID: \"df9bbdb1-5a45-4223-b03a-5d5e11542e60\") " pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-ktzwz"
Oct 30 22:04:09.681498 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:09.681465    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/df9bbdb1-5a45-4223-b03a-5d5e11542e60-istio-token\") pod \"e2e-worker-w2-0c99b1-openshift-default-967855977-ktzwz\" (UID: \"df9bbdb1-5a45-4223-b03a-5d5e11542e60\") " pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-ktzwz"
Oct 30 22:04:09.681694 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:09.681535    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t9px6\" (UniqueName: \"kubernetes.io/projected/df9bbdb1-5a45-4223-b03a-5d5e11542e60-kube-api-access-t9px6\") pod \"e2e-worker-w2-0c99b1-openshift-default-967855977-ktzwz\" (UID: \"df9bbdb1-5a45-4223-b03a-5d5e11542e60\") " pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-ktzwz"
Oct 30 22:04:09.946404 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:09.946304    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-ktzwz"
Oct 30 22:04:10.382050 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:10.382020    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-ktzwz"]
Oct 30 22:04:10.382419 ip-10-0-140-18 kubenswrapper[4473]: W1030 22:04:10.382074    4473 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf9bbdb1_5a45_4223_b03a_5d5e11542e60.slice/crio-71ee30e79eb3ff3e298888b70da5af551ad470cef89edb23c63afaae6b527b8f WatchSource:0}: Error finding container 71ee30e79eb3ff3e298888b70da5af551ad470cef89edb23c63afaae6b527b8f: Status 404 returned error can't find the container with id 71ee30e79eb3ff3e298888b70da5af551ad470cef89edb23c63afaae6b527b8f
Oct 30 22:04:10.384281 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:10.384241    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:04:10.384416 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:10.384314    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:04:10.384416 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:10.384341    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:04:10.802516 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:10.801335    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-ktzwz" event={"ID":"df9bbdb1-5a45-4223-b03a-5d5e11542e60","Type":"ContainerStarted","Data":"db07e6e36250d9ebf8ae105051d349e432a4d0ba0d0eaecfe66b14d4a53c972c"}
Oct 30 22:04:10.802516 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:10.801400    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-ktzwz" event={"ID":"df9bbdb1-5a45-4223-b03a-5d5e11542e60","Type":"ContainerStarted","Data":"71ee30e79eb3ff3e298888b70da5af551ad470cef89edb23c63afaae6b527b8f"}
Oct 30 22:04:10.970015 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:10.957719    4473 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-ktzwz" podStartSLOduration=1.957693172 podStartE2EDuration="1.957693172s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:04:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:04:10.93062498 +0000 UTC m=+1296.703993468" watchObservedRunningTime="2026-10-30 22:04:10.957693172 +0000 UTC m=+1296.731061662"
Oct 30 22:04:11.269004 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:11.238673    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-ktzwz"
Oct 30 22:04:11.287946 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:11.286114    4473 patch_prober.go:29] interesting pod/e2e-worker-w2-0c99b1-openshift-default-967855977-ktzwz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused" start-of-body=
Oct 30 22:04:11.287946 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:11.286215    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-ktzwz" podUID="df9bbdb1-5a45-4223-b03a-5d5e11542e60" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused"
Oct 30 22:04:11.948948 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:11.948035    4473 patch_prober.go:29] interesting pod/e2e-worker-w2-0c99b1-openshift-default-967855977-ktzwz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused" start-of-body=
Oct 30 22:04:11.948948 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:11.948102    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-ktzwz" podUID="df9bbdb1-5a45-4223-b03a-5d5e11542e60" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused"
Oct 30 22:04:12.138954 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:12.138852    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-dde1e8/distinct-2-w0-dde1e8-kserve-744857d57-ftt8b" podUID="577b020d-2a6b-4020-863c-cf19d4f0ca3b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.79:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:04:12.180944 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:12.178085    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-9f8aea/distinct-w5-9f8aea-kserve-5b564579cd-kmxzz" podUID="b11e8b11-334f-4a9e-98c2-b0164567a8af" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:04:12.245016 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:12.244889    4473 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-0f42cb/distinct-2-w3-0f42cb-kserve-fc477897b-jk8qc" podUID="61b49182-057e-41af-b609-1a6abf82c197" containerName="main" probeResult="failure" output="Get \"https://10.133.0.80:8000/health\": context deadline exceeded"
Oct 30 22:04:12.264958 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:12.256455    4473 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-0f42cb/distinct-w3-0f42cb-kserve-fb54fdd59-n6w4l" podUID="ff3c2b25-08aa-4b70-bf57-9cf4a11d05ed" containerName="main" probeResult="failure" output="Get \"https://10.133.0.73:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:04:12.290313 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:12.290263    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-0c99b1/distinct-2-w2-0c99b1-kserve-7466864658-8z7fk" podUID="89c09555-cf53-49a7-9c3b-95d2da2a293a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.82:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:04:12.585079 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:12.578019    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-29cbad/premium-w6-29cbad-kserve-6c9f8d6899-qtv6c" podUID="381b11f6-3fde-4de7-9ad3-7c0891e76ba1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:04:12.950974 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:12.950403    4473 patch_prober.go:29] interesting pod/e2e-worker-w2-0c99b1-openshift-default-967855977-ktzwz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused" start-of-body=
Oct 30 22:04:12.950974 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:12.950482    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-ktzwz" podUID="df9bbdb1-5a45-4223-b03a-5d5e11542e60" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused"
Oct 30 22:04:13.087945 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:13.087041    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-0f42cb/distinct-w3-0f42cb-kserve-fb54fdd59-n6w4l" podUID="ff3c2b25-08aa-4b70-bf57-9cf4a11d05ed" containerName="main" probeResult="failure" output="Get \"https://10.133.0.73:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:04:13.141949 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:13.141306    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-dde1e8/distinct-2-w0-dde1e8-kserve-744857d57-ftt8b" podUID="577b020d-2a6b-4020-863c-cf19d4f0ca3b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.79:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:04:13.145947 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:13.142978    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-dde1e8/distinct-2-w0-dde1e8-kserve-744857d57-ftt8b"
Oct 30 22:04:13.197948 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:13.192070    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-9f8aea/distinct-w5-9f8aea-kserve-5b564579cd-kmxzz" podUID="b11e8b11-334f-4a9e-98c2-b0164567a8af" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": context deadline exceeded"
Oct 30 22:04:13.197948 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:13.192598    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-9f8aea/distinct-w5-9f8aea-kserve-5b564579cd-kmxzz"
Oct 30 22:04:13.325008 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:13.312085    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-0c99b1/distinct-2-w2-0c99b1-kserve-7466864658-8z7fk" podUID="89c09555-cf53-49a7-9c3b-95d2da2a293a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.82:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:04:13.325008 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:13.312633    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-0c99b1/distinct-2-w2-0c99b1-kserve-7466864658-8z7fk"
Oct 30 22:04:13.586016 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:13.583666    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-29cbad/premium-w6-29cbad-kserve-6c9f8d6899-qtv6c" podUID="381b11f6-3fde-4de7-9ad3-7c0891e76ba1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:04:13.586016 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:13.584452    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-29cbad/premium-w6-29cbad-kserve-6c9f8d6899-qtv6c"
Oct 30 22:04:13.822947 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:13.814310    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-dde1e8/distinct-w0-dde1e8-kserve-75b68b4c5d-hts2k" podUID="9d72e7c9-657e-4278-8ed9-53adb75bd530" containerName="main" probeResult="failure" output="Get \"https://10.133.0.72:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:04:13.824274 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:13.824228    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-0f42cb/distinct-2-w3-0f42cb-kserve-fc477897b-jk8qc" podUID="61b49182-057e-41af-b609-1a6abf82c197" containerName="main" probeResult="failure" output="Get \"https://10.133.0.80:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:04:13.878159 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:13.878021    4473 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-dde1e8/distinct-w0-dde1e8-kserve-75b68b4c5d-hts2k" podUID="9d72e7c9-657e-4278-8ed9-53adb75bd530" containerName="main" probeResult="failure" output="Get \"https://10.133.0.72:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:04:14.213500 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:14.213402    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-9f8aea/distinct-w5-9f8aea-kserve-5b564579cd-kmxzz" podUID="b11e8b11-334f-4a9e-98c2-b0164567a8af" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:04:14.325950 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:14.278793    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-0f42cb/distinct-w3-0f42cb-kserve-fb54fdd59-n6w4l" podUID="ff3c2b25-08aa-4b70-bf57-9cf4a11d05ed" containerName="main" probeResult="failure" output="Get \"https://10.133.0.73:8000/ready\": context deadline exceeded"
Oct 30 22:04:14.325950 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:14.279123    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-dde1e8/distinct-2-w0-dde1e8-kserve-744857d57-ftt8b" podUID="577b020d-2a6b-4020-863c-cf19d4f0ca3b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.79:8000/ready\": context deadline exceeded"
Oct 30 22:04:14.325950 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:14.311285    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-0f42cb/distinct-w3-0f42cb-kserve-fb54fdd59-n6w4l"
Oct 30 22:04:14.325950 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:14.314307    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-0c99b1/distinct-2-w2-0c99b1-kserve-7466864658-8z7fk" podUID="89c09555-cf53-49a7-9c3b-95d2da2a293a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.82:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:04:14.507010 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:14.463761    4473 patch_prober.go:29] interesting pod/e2e-worker-w2-0c99b1-openshift-default-967855977-ktzwz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused" start-of-body=
Oct 30 22:04:14.620401 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:14.620362    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-ktzwz" podUID="df9bbdb1-5a45-4223-b03a-5d5e11542e60" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused"
Oct 30 22:04:14.621026 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:14.583876    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-29cbad/premium-w6-29cbad-kserve-6c9f8d6899-qtv6c" podUID="381b11f6-3fde-4de7-9ad3-7c0891e76ba1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:04:14.852302 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:14.852059    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-0f42cb/distinct-2-w3-0f42cb-kserve-fc477897b-jk8qc" podUID="61b49182-057e-41af-b609-1a6abf82c197" containerName="main" probeResult="failure" output="Get \"https://10.133.0.80:8000/ready\": context deadline exceeded"
Oct 30 22:04:14.852841 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:14.852822    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-0f42cb/distinct-2-w3-0f42cb-kserve-fc477897b-jk8qc"
Oct 30 22:04:14.958845 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:14.958813    4473 patch_prober.go:29] interesting pod/e2e-worker-w2-0c99b1-openshift-default-967855977-ktzwz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused" start-of-body=
Oct 30 22:04:14.959178 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:14.959131    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-ktzwz" podUID="df9bbdb1-5a45-4223-b03a-5d5e11542e60" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused"
Oct 30 22:04:15.235948 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:15.223236    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-9f8aea/distinct-w5-9f8aea-kserve-5b564579cd-kmxzz" podUID="b11e8b11-334f-4a9e-98c2-b0164567a8af" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:04:15.316960 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:15.313235    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-dde1e8/distinct-2-w0-dde1e8-kserve-744857d57-ftt8b" podUID="577b020d-2a6b-4020-863c-cf19d4f0ca3b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.79:8000/ready\": context deadline exceeded"
Oct 30 22:04:15.316960 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:15.313668    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-0f42cb/distinct-w3-0f42cb-kserve-fb54fdd59-n6w4l" podUID="ff3c2b25-08aa-4b70-bf57-9cf4a11d05ed" containerName="main" probeResult="failure" output="Get \"https://10.133.0.73:8000/ready\": context deadline exceeded"
Oct 30 22:04:15.316960 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:15.314533    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-0c99b1/distinct-2-w2-0c99b1-kserve-7466864658-8z7fk" podUID="89c09555-cf53-49a7-9c3b-95d2da2a293a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.82:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:04:15.328029 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:15.319569    4473 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-dde1e8/distinct-2-w0-dde1e8-kserve-744857d57-ftt8b" podUID="577b020d-2a6b-4020-863c-cf19d4f0ca3b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.79:8000/health\": context deadline exceeded"
Oct 30 22:04:15.653950 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:15.650033    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-29cbad/premium-w6-29cbad-kserve-6c9f8d6899-qtv6c" podUID="381b11f6-3fde-4de7-9ad3-7c0891e76ba1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:04:15.755214 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:15.755166    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-dde1e8/distinct-w0-dde1e8-kserve-75b68b4c5d-hts2k" podUID="9d72e7c9-657e-4278-8ed9-53adb75bd530" containerName="main" probeResult="failure" output="Get \"https://10.133.0.72:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:04:15.860950 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:15.855023    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-0f42cb/distinct-2-w3-0f42cb-kserve-fc477897b-jk8qc" podUID="61b49182-057e-41af-b609-1a6abf82c197" containerName="main" probeResult="failure" output="Get \"https://10.133.0.80:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:04:15.949570 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:15.949066    4473 patch_prober.go:29] interesting pod/e2e-worker-w2-0c99b1-openshift-default-967855977-ktzwz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused" start-of-body=
Oct 30 22:04:15.949570 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:15.949130    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-ktzwz" podUID="df9bbdb1-5a45-4223-b03a-5d5e11542e60" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused"
Oct 30 22:04:16.093952 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:16.083252    4473 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w5-9f8aea/distinct-w5-9f8aea-kserve-5b564579cd-kmxzz" podUID="b11e8b11-334f-4a9e-98c2-b0164567a8af" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:04:16.227087 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:16.226049    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-9f8aea/distinct-w5-9f8aea-kserve-5b564579cd-kmxzz" podUID="b11e8b11-334f-4a9e-98c2-b0164567a8af" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:04:16.337678 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:16.337620    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-0c99b1/distinct-2-w2-0c99b1-kserve-7466864658-8z7fk" podUID="89c09555-cf53-49a7-9c3b-95d2da2a293a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.82:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:04:16.367229 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:16.338726    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-dde1e8/distinct-2-w0-dde1e8-kserve-744857d57-ftt8b" podUID="577b020d-2a6b-4020-863c-cf19d4f0ca3b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.79:8000/ready\": context deadline exceeded"
Oct 30 22:04:16.367815 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:16.367773    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-0f42cb/distinct-w3-0f42cb-kserve-fb54fdd59-n6w4l" podUID="ff3c2b25-08aa-4b70-bf57-9cf4a11d05ed" containerName="main" probeResult="failure" output="Get \"https://10.133.0.73:8000/ready\": context deadline exceeded"
Oct 30 22:04:16.679601 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:16.679550    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-29cbad/premium-w6-29cbad-kserve-6c9f8d6899-qtv6c" podUID="381b11f6-3fde-4de7-9ad3-7c0891e76ba1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.60:8000/ready\": context deadline exceeded"
Oct 30 22:04:16.805690 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:16.805644    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-dde1e8/distinct-w0-dde1e8-kserve-75b68b4c5d-hts2k" podUID="9d72e7c9-657e-4278-8ed9-53adb75bd530" containerName="main" probeResult="failure" output="Get \"https://10.133.0.72:8000/ready\": context deadline exceeded"
Oct 30 22:04:17.013005 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:17.010717    4473 patch_prober.go:29] interesting pod/e2e-worker-w2-0c99b1-openshift-default-967855977-ktzwz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused" start-of-body=
Oct 30 22:04:17.013005 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:17.010788    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-ktzwz" podUID="df9bbdb1-5a45-4223-b03a-5d5e11542e60" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused"
Oct 30 22:04:17.269475 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:17.269351    4473 patch_prober.go:29] interesting pod/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-jvp7n container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 22:04:17.269761 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:17.269713    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-jvp7n" podUID="091485c0-17be-49be-a5bc-a26a9b2e89a3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:04:17.319063 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:17.319006    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-0f42cb/distinct-2-w3-0f42cb-kserve-fc477897b-jk8qc" podUID="61b49182-057e-41af-b609-1a6abf82c197" containerName="main" probeResult="failure" output="Get \"https://10.133.0.80:8000/ready\": context deadline exceeded"
Oct 30 22:04:17.375256 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:17.369061    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-9f8aea/distinct-w5-9f8aea-kserve-5b564579cd-kmxzz" podUID="b11e8b11-334f-4a9e-98c2-b0164567a8af" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:04:17.740948 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:04:17.737481    4473 kubelet.go:2711] "Housekeeping took longer than expected" err="housekeeping took too long" expected="1s" actual="1.258s"
Oct 30 22:04:17.741388 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:17.741365    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-dde1e8/distinct-w0-dde1e8-kserve-75b68b4c5d-hts2k"
Oct 30 22:04:17.804949 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:17.784578    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-0c99b1/distinct-2-w2-0c99b1-kserve-7466864658-8z7fk" podUID="89c09555-cf53-49a7-9c3b-95d2da2a293a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.82:8000/ready\": context deadline exceeded"
Oct 30 22:04:17.804949 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:17.784850    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-dde1e8/distinct-2-w0-dde1e8-kserve-744857d57-ftt8b" podUID="577b020d-2a6b-4020-863c-cf19d4f0ca3b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.79:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:04:17.804949 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:17.785099    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-0f42cb/distinct-w3-0f42cb-kserve-fb54fdd59-n6w4l" podUID="ff3c2b25-08aa-4b70-bf57-9cf4a11d05ed" containerName="main" probeResult="failure" output="Get \"https://10.133.0.73:8000/ready\": context deadline exceeded"
Oct 30 22:04:17.893408 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:17.893368    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-29cbad/premium-w6-29cbad-kserve-6c9f8d6899-qtv6c" podUID="381b11f6-3fde-4de7-9ad3-7c0891e76ba1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.60:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:04:17.978097 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:17.976809    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/echo-simulator-kserve-64b6f8f96c-m6tlf" podUID="e9ca7766-41cc-4042-ae52-ce6451f31d4b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.37:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:04:17.995643 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:17.995560    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-0c99b1/distinct-2-w2-0c99b1-kserve-7466864658-8z7fk"
Oct 30 22:04:18.022158 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:18.016056    4473 patch_prober.go:29] interesting pod/e2e-worker-w2-0c99b1-openshift-default-967855977-ktzwz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused" start-of-body=
Oct 30 22:04:18.022158 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:18.016145    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-ktzwz" podUID="df9bbdb1-5a45-4223-b03a-5d5e11542e60" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused"
Oct 30 22:04:18.107954 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:18.104160    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-dde1e8/distinct-w0-dde1e8-kserve-75b68b4c5d-hts2k" podUID="9d72e7c9-657e-4278-8ed9-53adb75bd530" containerName="main" probeResult="failure" output="Get \"https://10.133.0.72:8000/ready\": context deadline exceeded"
Oct 30 22:04:18.112551 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:18.112527    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-dde1e8/distinct-2-w0-dde1e8-kserve-744857d57-ftt8b"
Oct 30 22:04:18.112813 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:18.112795    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-9f8aea/distinct-w5-9f8aea-kserve-5b564579cd-kmxzz"
Oct 30 22:04:18.113013 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:18.113003    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-29cbad/premium-w6-29cbad-kserve-6c9f8d6899-qtv6c"
Oct 30 22:04:18.116857 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:18.116742    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-0f42cb/distinct-2-w3-0f42cb-kserve-fc477897b-jk8qc"
Oct 30 22:04:18.160007 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:18.129108    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-dde1e8/distinct-w0-dde1e8-kserve-75b68b4c5d-hts2k"
Oct 30 22:04:18.162612 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:18.162586    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-0f42cb/distinct-w3-0f42cb-kserve-fb54fdd59-n6w4l"
Oct 30 22:04:18.950458 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:18.950424    4473 patch_prober.go:29] interesting pod/e2e-worker-w2-0c99b1-openshift-default-967855977-ktzwz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused" start-of-body=
Oct 30 22:04:18.951088 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:18.951060    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-ktzwz" podUID="df9bbdb1-5a45-4223-b03a-5d5e11542e60" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused"
Oct 30 22:04:19.950955 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:19.950339    4473 patch_prober.go:29] interesting pod/e2e-worker-w2-0c99b1-openshift-default-967855977-ktzwz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused" start-of-body=
Oct 30 22:04:19.950955 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:19.950402    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-ktzwz" podUID="df9bbdb1-5a45-4223-b03a-5d5e11542e60" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused"
Oct 30 22:04:20.356453 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:04:20.356367    4473 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fncvg" containerName="extractor"
Oct 30 22:04:20.953137 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:20.952208    4473 patch_prober.go:29] interesting pod/e2e-worker-w2-0c99b1-openshift-default-967855977-ktzwz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused" start-of-body=
Oct 30 22:04:20.953137 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:20.952276    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-ktzwz" podUID="df9bbdb1-5a45-4223-b03a-5d5e11542e60" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused"
Oct 30 22:04:21.946687 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:21.946648    4473 patch_prober.go:29] interesting pod/e2e-worker-w2-0c99b1-openshift-default-967855977-ktzwz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused" start-of-body=
Oct 30 22:04:21.946967 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:21.946732    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-ktzwz" podUID="df9bbdb1-5a45-4223-b03a-5d5e11542e60" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused"
Oct 30 22:04:22.952285 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:22.952250    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-ktzwz"
Oct 30 22:04:22.952784 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:22.952671    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-ktzwz"
Oct 30 22:04:22.953606 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:22.953586    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-ktzwz"
Oct 30 22:04:35.380178 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:35.380143    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-wbzjg"]
Oct 30 22:04:35.380672 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:35.380405    4473 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-wbzjg" podUID="4c4ef828-d895-488c-900d-3525e3fda397" containerName="istio-proxy" containerID="cri-o://ec24f45fc0eb542885d4c0e99a28f87054f4b6ec7879bbfb16be4742a91881d0" gracePeriod=30
Oct 30 22:04:40.675481 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:40.675454    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-wbzjg"
Oct 30 22:04:40.743536 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:40.743489    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4c4ef828-d895-488c-900d-3525e3fda397-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4c4ef828-d895-488c-900d-3525e3fda397-istiod-ca-cert\") pod \"4c4ef828-d895-488c-900d-3525e3fda397\" (UID: \"4c4ef828-d895-488c-900d-3525e3fda397\") "
Oct 30 22:04:40.743713 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:40.743551    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/4c4ef828-d895-488c-900d-3525e3fda397-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4c4ef828-d895-488c-900d-3525e3fda397-istio-podinfo\") pod \"4c4ef828-d895-488c-900d-3525e3fda397\" (UID: \"4c4ef828-d895-488c-900d-3525e3fda397\") "
Oct 30 22:04:40.743713 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:40.743666    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4c4ef828-d895-488c-900d-3525e3fda397-istio-token\" (UniqueName: \"kubernetes.io/projected/4c4ef828-d895-488c-900d-3525e3fda397-istio-token\") pod \"4c4ef828-d895-488c-900d-3525e3fda397\" (UID: \"4c4ef828-d895-488c-900d-3525e3fda397\") "
Oct 30 22:04:40.743789 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:40.743753    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4c4ef828-d895-488c-900d-3525e3fda397-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4c4ef828-d895-488c-900d-3525e3fda397-istio-envoy\") pod \"4c4ef828-d895-488c-900d-3525e3fda397\" (UID: \"4c4ef828-d895-488c-900d-3525e3fda397\") "
Oct 30 22:04:40.743904 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:40.743890    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4c4ef828-d895-488c-900d-3525e3fda397-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c4ef828-d895-488c-900d-3525e3fda397-workload-socket\") pod \"4c4ef828-d895-488c-900d-3525e3fda397\" (UID: \"4c4ef828-d895-488c-900d-3525e3fda397\") "
Oct 30 22:04:40.743979 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:40.743943    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4c4ef828-d895-488c-900d-3525e3fda397-istiod-ca-cert" pod "4c4ef828-d895-488c-900d-3525e3fda397" (UID: "4c4ef828-d895-488c-900d-3525e3fda397"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:04:40.743979 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:40.743961    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4c4ef828-d895-488c-900d-3525e3fda397-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4c4ef828-d895-488c-900d-3525e3fda397-workload-certs\") pod \"4c4ef828-d895-488c-900d-3525e3fda397\" (UID: \"4c4ef828-d895-488c-900d-3525e3fda397\") "
Oct 30 22:04:40.744163 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:40.744144    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c4ef828-d895-488c-900d-3525e3fda397-workload-certs" pod "4c4ef828-d895-488c-900d-3525e3fda397" (UID: "4c4ef828-d895-488c-900d-3525e3fda397"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:04:40.744235 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:40.744158    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c4ef828-d895-488c-900d-3525e3fda397-workload-socket" pod "4c4ef828-d895-488c-900d-3525e3fda397" (UID: "4c4ef828-d895-488c-900d-3525e3fda397"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:04:40.744371 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:40.744260    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4c4ef828-d895-488c-900d-3525e3fda397-kube-api-access-rp9bs\" (UniqueName: \"kubernetes.io/projected/4c4ef828-d895-488c-900d-3525e3fda397-kube-api-access-rp9bs\") pod \"4c4ef828-d895-488c-900d-3525e3fda397\" (UID: \"4c4ef828-d895-488c-900d-3525e3fda397\") "
Oct 30 22:04:40.744371 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:40.744371    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4c4ef828-d895-488c-900d-3525e3fda397-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c4ef828-d895-488c-900d-3525e3fda397-credential-socket\") pod \"4c4ef828-d895-488c-900d-3525e3fda397\" (UID: \"4c4ef828-d895-488c-900d-3525e3fda397\") "
Oct 30 22:04:40.744565 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:40.744421    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4c4ef828-d895-488c-900d-3525e3fda397-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4c4ef828-d895-488c-900d-3525e3fda397-istio-data\") pod \"4c4ef828-d895-488c-900d-3525e3fda397\" (UID: \"4c4ef828-d895-488c-900d-3525e3fda397\") "
Oct 30 22:04:40.744691 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:40.744670    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c4ef828-d895-488c-900d-3525e3fda397-credential-socket" pod "4c4ef828-d895-488c-900d-3525e3fda397" (UID: "4c4ef828-d895-488c-900d-3525e3fda397"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:04:40.744905 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:40.744875    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c4ef828-d895-488c-900d-3525e3fda397-istio-data" pod "4c4ef828-d895-488c-900d-3525e3fda397" (UID: "4c4ef828-d895-488c-900d-3525e3fda397"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:04:40.745181 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:40.745162    4473 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c4ef828-d895-488c-900d-3525e3fda397-credential-socket\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:04:40.745271 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:40.745188    4473 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4c4ef828-d895-488c-900d-3525e3fda397-istio-data\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:04:40.745271 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:40.745212    4473 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4c4ef828-d895-488c-900d-3525e3fda397-istiod-ca-cert\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:04:40.745271 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:40.745226    4473 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c4ef828-d895-488c-900d-3525e3fda397-workload-socket\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:04:40.745271 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:40.745239    4473 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4c4ef828-d895-488c-900d-3525e3fda397-workload-certs\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:04:40.746177 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:40.746152    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c4ef828-d895-488c-900d-3525e3fda397-istio-envoy" pod "4c4ef828-d895-488c-900d-3525e3fda397" (UID: "4c4ef828-d895-488c-900d-3525e3fda397"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:04:40.746293 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:40.746220    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4c4ef828-d895-488c-900d-3525e3fda397-istio-token" pod "4c4ef828-d895-488c-900d-3525e3fda397" (UID: "4c4ef828-d895-488c-900d-3525e3fda397"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:04:40.746510 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:40.746489    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4c4ef828-d895-488c-900d-3525e3fda397-istio-podinfo" pod "4c4ef828-d895-488c-900d-3525e3fda397" (UID: "4c4ef828-d895-488c-900d-3525e3fda397"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:04:40.746746 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:40.746726    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4c4ef828-d895-488c-900d-3525e3fda397-kube-api-access-rp9bs" pod "4c4ef828-d895-488c-900d-3525e3fda397" (UID: "4c4ef828-d895-488c-900d-3525e3fda397"). InnerVolumeSpecName "kube-api-access-rp9bs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:04:40.834561 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:40.834524    4473 generic.go:356] "Generic (PLEG): container finished" podID="4c4ef828-d895-488c-900d-3525e3fda397" containerID="ec24f45fc0eb542885d4c0e99a28f87054f4b6ec7879bbfb16be4742a91881d0" exitCode=0
Oct 30 22:04:40.834811 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:40.834596    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-wbzjg" event={"ID":"4c4ef828-d895-488c-900d-3525e3fda397","Type":"ContainerDied","Data":"ec24f45fc0eb542885d4c0e99a28f87054f4b6ec7879bbfb16be4742a91881d0"}
Oct 30 22:04:40.834811 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:40.834614    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-wbzjg"
Oct 30 22:04:40.834811 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:40.834692    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-wbzjg" event={"ID":"4c4ef828-d895-488c-900d-3525e3fda397","Type":"ContainerDied","Data":"46b5504379e0e10585bfb5292d5912fb2e3cba5a6a197c5be0d576382baf482a"}
Oct 30 22:04:40.834811 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:40.834718    4473 scope.go:122] "RemoveContainer" containerID="ec24f45fc0eb542885d4c0e99a28f87054f4b6ec7879bbfb16be4742a91881d0"
Oct 30 22:04:40.846580 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:40.846554    4473 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4c4ef828-d895-488c-900d-3525e3fda397-istio-podinfo\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:04:40.846697 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:40.846584    4473 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4c4ef828-d895-488c-900d-3525e3fda397-istio-token\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:04:40.846697 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:40.846598    4473 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4c4ef828-d895-488c-900d-3525e3fda397-istio-envoy\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:04:40.846697 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:40.846637    4473 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rp9bs\" (UniqueName: \"kubernetes.io/projected/4c4ef828-d895-488c-900d-3525e3fda397-kube-api-access-rp9bs\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:04:40.857639 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:40.857589    4473 scope.go:122] "RemoveContainer" containerID="ec24f45fc0eb542885d4c0e99a28f87054f4b6ec7879bbfb16be4742a91881d0"
Oct 30 22:04:40.858060 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:04:40.858037    4473 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ec24f45fc0eb542885d4c0e99a28f87054f4b6ec7879bbfb16be4742a91881d0\": container with ID starting with ec24f45fc0eb542885d4c0e99a28f87054f4b6ec7879bbfb16be4742a91881d0 not found: ID does not exist" containerID="ec24f45fc0eb542885d4c0e99a28f87054f4b6ec7879bbfb16be4742a91881d0"
Oct 30 22:04:40.858157 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:40.858074    4473 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ec24f45fc0eb542885d4c0e99a28f87054f4b6ec7879bbfb16be4742a91881d0"} err="failed to get container status \"ec24f45fc0eb542885d4c0e99a28f87054f4b6ec7879bbfb16be4742a91881d0\": rpc error: code = NotFound desc = could not find container \"ec24f45fc0eb542885d4c0e99a28f87054f4b6ec7879bbfb16be4742a91881d0\": container with ID starting with ec24f45fc0eb542885d4c0e99a28f87054f4b6ec7879bbfb16be4742a91881d0 not found: ID does not exist"
Oct 30 22:04:41.041259 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:41.041164    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-wbzjg"]
Oct 30 22:04:41.047068 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:41.047037    4473 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-wbzjg"]
Oct 30 22:04:41.300156 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:41.300074    4473 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-8cv5d"]
Oct 30 22:04:41.300804 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:41.300789    4473 cpu_manager.go:414] "containerMap: removing container" podUID="4c4ef828-d895-488c-900d-3525e3fda397" containerName="istio-proxy"
Oct 30 22:04:41.300847 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:41.300805    4473 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4c4ef828-d895-488c-900d-3525e3fda397" containerName="istio-proxy"
Oct 30 22:04:41.300956 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:41.300909    4473 memory_manager.go:365] "RemoveStaleState removing state" podUID="4c4ef828-d895-488c-900d-3525e3fda397" containerName="istio-proxy"
Oct 30 22:04:41.306094 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:41.306068    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-8cv5d"
Oct 30 22:04:41.353573 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:41.353532    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/85b6bb64-5545-4a61-bb0d-a4851348fe30-istio-token\") pod \"e2e-worker-w3-0f42cb-openshift-default-584f4c4865-8cv5d\" (UID: \"85b6bb64-5545-4a61-bb0d-a4851348fe30\") " pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-8cv5d"
Oct 30 22:04:41.353725 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:41.353584    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d42z4\" (UniqueName: \"kubernetes.io/projected/85b6bb64-5545-4a61-bb0d-a4851348fe30-kube-api-access-d42z4\") pod \"e2e-worker-w3-0f42cb-openshift-default-584f4c4865-8cv5d\" (UID: \"85b6bb64-5545-4a61-bb0d-a4851348fe30\") " pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-8cv5d"
Oct 30 22:04:41.353848 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:41.353823    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/85b6bb64-5545-4a61-bb0d-a4851348fe30-workload-certs\") pod \"e2e-worker-w3-0f42cb-openshift-default-584f4c4865-8cv5d\" (UID: \"85b6bb64-5545-4a61-bb0d-a4851348fe30\") " pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-8cv5d"
Oct 30 22:04:41.353903 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:41.353864    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/85b6bb64-5545-4a61-bb0d-a4851348fe30-istiod-ca-cert\") pod \"e2e-worker-w3-0f42cb-openshift-default-584f4c4865-8cv5d\" (UID: \"85b6bb64-5545-4a61-bb0d-a4851348fe30\") " pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-8cv5d"
Oct 30 22:04:41.354055 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:41.354036    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/85b6bb64-5545-4a61-bb0d-a4851348fe30-istio-podinfo\") pod \"e2e-worker-w3-0f42cb-openshift-default-584f4c4865-8cv5d\" (UID: \"85b6bb64-5545-4a61-bb0d-a4851348fe30\") " pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-8cv5d"
Oct 30 22:04:41.354119 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:41.354072    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/85b6bb64-5545-4a61-bb0d-a4851348fe30-istio-envoy\") pod \"e2e-worker-w3-0f42cb-openshift-default-584f4c4865-8cv5d\" (UID: \"85b6bb64-5545-4a61-bb0d-a4851348fe30\") " pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-8cv5d"
Oct 30 22:04:41.354171 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:41.354125    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/85b6bb64-5545-4a61-bb0d-a4851348fe30-istio-data\") pod \"e2e-worker-w3-0f42cb-openshift-default-584f4c4865-8cv5d\" (UID: \"85b6bb64-5545-4a61-bb0d-a4851348fe30\") " pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-8cv5d"
Oct 30 22:04:41.354224 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:41.354203    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/85b6bb64-5545-4a61-bb0d-a4851348fe30-workload-socket\") pod \"e2e-worker-w3-0f42cb-openshift-default-584f4c4865-8cv5d\" (UID: \"85b6bb64-5545-4a61-bb0d-a4851348fe30\") " pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-8cv5d"
Oct 30 22:04:41.354336 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:41.354320    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/85b6bb64-5545-4a61-bb0d-a4851348fe30-credential-socket\") pod \"e2e-worker-w3-0f42cb-openshift-default-584f4c4865-8cv5d\" (UID: \"85b6bb64-5545-4a61-bb0d-a4851348fe30\") " pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-8cv5d"
Oct 30 22:04:41.382002 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:41.381955    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-8cv5d"]
Oct 30 22:04:41.455688 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:41.455653    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/85b6bb64-5545-4a61-bb0d-a4851348fe30-credential-socket\") pod \"e2e-worker-w3-0f42cb-openshift-default-584f4c4865-8cv5d\" (UID: \"85b6bb64-5545-4a61-bb0d-a4851348fe30\") " pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-8cv5d"
Oct 30 22:04:41.455853 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:41.455798    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/85b6bb64-5545-4a61-bb0d-a4851348fe30-istio-token\") pod \"e2e-worker-w3-0f42cb-openshift-default-584f4c4865-8cv5d\" (UID: \"85b6bb64-5545-4a61-bb0d-a4851348fe30\") " pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-8cv5d"
Oct 30 22:04:41.455853 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:41.455825    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d42z4\" (UniqueName: \"kubernetes.io/projected/85b6bb64-5545-4a61-bb0d-a4851348fe30-kube-api-access-d42z4\") pod \"e2e-worker-w3-0f42cb-openshift-default-584f4c4865-8cv5d\" (UID: \"85b6bb64-5545-4a61-bb0d-a4851348fe30\") " pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-8cv5d"
Oct 30 22:04:41.456125 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:41.456106    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/85b6bb64-5545-4a61-bb0d-a4851348fe30-workload-certs\") pod \"e2e-worker-w3-0f42cb-openshift-default-584f4c4865-8cv5d\" (UID: \"85b6bb64-5545-4a61-bb0d-a4851348fe30\") " pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-8cv5d"
Oct 30 22:04:41.456224 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:41.456106    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/85b6bb64-5545-4a61-bb0d-a4851348fe30-credential-socket\") pod \"e2e-worker-w3-0f42cb-openshift-default-584f4c4865-8cv5d\" (UID: \"85b6bb64-5545-4a61-bb0d-a4851348fe30\") " pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-8cv5d"
Oct 30 22:04:41.456224 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:41.456149    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/85b6bb64-5545-4a61-bb0d-a4851348fe30-istiod-ca-cert\") pod \"e2e-worker-w3-0f42cb-openshift-default-584f4c4865-8cv5d\" (UID: \"85b6bb64-5545-4a61-bb0d-a4851348fe30\") " pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-8cv5d"
Oct 30 22:04:41.456342 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:41.456254    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/85b6bb64-5545-4a61-bb0d-a4851348fe30-istio-podinfo\") pod \"e2e-worker-w3-0f42cb-openshift-default-584f4c4865-8cv5d\" (UID: \"85b6bb64-5545-4a61-bb0d-a4851348fe30\") " pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-8cv5d"
Oct 30 22:04:41.456342 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:41.456280    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/85b6bb64-5545-4a61-bb0d-a4851348fe30-istio-envoy\") pod \"e2e-worker-w3-0f42cb-openshift-default-584f4c4865-8cv5d\" (UID: \"85b6bb64-5545-4a61-bb0d-a4851348fe30\") " pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-8cv5d"
Oct 30 22:04:41.456342 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:41.456296    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/85b6bb64-5545-4a61-bb0d-a4851348fe30-istio-data\") pod \"e2e-worker-w3-0f42cb-openshift-default-584f4c4865-8cv5d\" (UID: \"85b6bb64-5545-4a61-bb0d-a4851348fe30\") " pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-8cv5d"
Oct 30 22:04:41.456342 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:41.456313    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/85b6bb64-5545-4a61-bb0d-a4851348fe30-workload-socket\") pod \"e2e-worker-w3-0f42cb-openshift-default-584f4c4865-8cv5d\" (UID: \"85b6bb64-5545-4a61-bb0d-a4851348fe30\") " pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-8cv5d"
Oct 30 22:04:41.456544 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:41.456342    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/85b6bb64-5545-4a61-bb0d-a4851348fe30-workload-certs\") pod \"e2e-worker-w3-0f42cb-openshift-default-584f4c4865-8cv5d\" (UID: \"85b6bb64-5545-4a61-bb0d-a4851348fe30\") " pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-8cv5d"
Oct 30 22:04:41.456609 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:41.456543    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/85b6bb64-5545-4a61-bb0d-a4851348fe30-workload-socket\") pod \"e2e-worker-w3-0f42cb-openshift-default-584f4c4865-8cv5d\" (UID: \"85b6bb64-5545-4a61-bb0d-a4851348fe30\") " pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-8cv5d"
Oct 30 22:04:41.456689 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:41.456670    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/85b6bb64-5545-4a61-bb0d-a4851348fe30-istio-data\") pod \"e2e-worker-w3-0f42cb-openshift-default-584f4c4865-8cv5d\" (UID: \"85b6bb64-5545-4a61-bb0d-a4851348fe30\") " pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-8cv5d"
Oct 30 22:04:41.456805 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:41.456787    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/85b6bb64-5545-4a61-bb0d-a4851348fe30-istiod-ca-cert\") pod \"e2e-worker-w3-0f42cb-openshift-default-584f4c4865-8cv5d\" (UID: \"85b6bb64-5545-4a61-bb0d-a4851348fe30\") " pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-8cv5d"
Oct 30 22:04:41.458791 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:41.458763    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/85b6bb64-5545-4a61-bb0d-a4851348fe30-istio-envoy\") pod \"e2e-worker-w3-0f42cb-openshift-default-584f4c4865-8cv5d\" (UID: \"85b6bb64-5545-4a61-bb0d-a4851348fe30\") " pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-8cv5d"
Oct 30 22:04:41.459026 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:41.459007    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/85b6bb64-5545-4a61-bb0d-a4851348fe30-istio-podinfo\") pod \"e2e-worker-w3-0f42cb-openshift-default-584f4c4865-8cv5d\" (UID: \"85b6bb64-5545-4a61-bb0d-a4851348fe30\") " pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-8cv5d"
Oct 30 22:04:41.514100 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:41.514065    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/85b6bb64-5545-4a61-bb0d-a4851348fe30-istio-token\") pod \"e2e-worker-w3-0f42cb-openshift-default-584f4c4865-8cv5d\" (UID: \"85b6bb64-5545-4a61-bb0d-a4851348fe30\") " pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-8cv5d"
Oct 30 22:04:41.514293 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:41.514270    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d42z4\" (UniqueName: \"kubernetes.io/projected/85b6bb64-5545-4a61-bb0d-a4851348fe30-kube-api-access-d42z4\") pod \"e2e-worker-w3-0f42cb-openshift-default-584f4c4865-8cv5d\" (UID: \"85b6bb64-5545-4a61-bb0d-a4851348fe30\") " pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-8cv5d"
Oct 30 22:04:41.622577 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:41.622530    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-8cv5d"
Oct 30 22:04:41.624548 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:41.624524    4473 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-dd67d"]
Oct 30 22:04:41.629248 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:41.629228    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-dd67d"
Oct 30 22:04:41.640558 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:41.640529    4473 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-pxzmr"]
Oct 30 22:04:41.645280 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:41.645258    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-pxzmr"
Oct 30 22:04:41.661121 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:41.660627    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9dc3b95a-13ab-453d-a5a5-d9037ebcfada-istio-envoy\") pod \"e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-dd67d\" (UID: \"9dc3b95a-13ab-453d-a5a5-d9037ebcfada\") " pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-dd67d"
Oct 30 22:04:41.661121 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:41.660676    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9dc3b95a-13ab-453d-a5a5-d9037ebcfada-credential-socket\") pod \"e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-dd67d\" (UID: \"9dc3b95a-13ab-453d-a5a5-d9037ebcfada\") " pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-dd67d"
Oct 30 22:04:41.661121 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:41.660873    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9dc3b95a-13ab-453d-a5a5-d9037ebcfada-istio-token\") pod \"e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-dd67d\" (UID: \"9dc3b95a-13ab-453d-a5a5-d9037ebcfada\") " pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-dd67d"
Oct 30 22:04:41.661121 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:41.661002    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9dc3b95a-13ab-453d-a5a5-d9037ebcfada-istio-data\") pod \"e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-dd67d\" (UID: \"9dc3b95a-13ab-453d-a5a5-d9037ebcfada\") " pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-dd67d"
Oct 30 22:04:41.661121 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:41.661042    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9dc3b95a-13ab-453d-a5a5-d9037ebcfada-istiod-ca-cert\") pod \"e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-dd67d\" (UID: \"9dc3b95a-13ab-453d-a5a5-d9037ebcfada\") " pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-dd67d"
Oct 30 22:04:41.661121 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:41.661094    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dxjxj\" (UniqueName: \"kubernetes.io/projected/9dc3b95a-13ab-453d-a5a5-d9037ebcfada-kube-api-access-dxjxj\") pod \"e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-dd67d\" (UID: \"9dc3b95a-13ab-453d-a5a5-d9037ebcfada\") " pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-dd67d"
Oct 30 22:04:41.662200 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:41.662172    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9dc3b95a-13ab-453d-a5a5-d9037ebcfada-istio-podinfo\") pod \"e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-dd67d\" (UID: \"9dc3b95a-13ab-453d-a5a5-d9037ebcfada\") " pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-dd67d"
Oct 30 22:04:41.663091 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:41.662241    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9dc3b95a-13ab-453d-a5a5-d9037ebcfada-workload-socket\") pod \"e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-dd67d\" (UID: \"9dc3b95a-13ab-453d-a5a5-d9037ebcfada\") " pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-dd67d"
Oct 30 22:04:41.663091 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:41.662393    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9dc3b95a-13ab-453d-a5a5-d9037ebcfada-workload-certs\") pod \"e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-dd67d\" (UID: \"9dc3b95a-13ab-453d-a5a5-d9037ebcfada\") " pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-dd67d"
Oct 30 22:04:41.698697 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:41.698664    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-dd67d"]
Oct 30 22:04:41.708072 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:41.707034    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-pxzmr"]
Oct 30 22:04:41.763696 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:41.763663    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3552416b-6da1-4c28-be4b-5450abc40f79-istio-data\") pod \"e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-pxzmr\" (UID: \"3552416b-6da1-4c28-be4b-5450abc40f79\") " pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-pxzmr"
Oct 30 22:04:41.763891 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:41.763720    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9dc3b95a-13ab-453d-a5a5-d9037ebcfada-workload-certs\") pod \"e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-dd67d\" (UID: \"9dc3b95a-13ab-453d-a5a5-d9037ebcfada\") " pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-dd67d"
Oct 30 22:04:41.763891 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:41.763870    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9dc3b95a-13ab-453d-a5a5-d9037ebcfada-istio-envoy\") pod \"e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-dd67d\" (UID: \"9dc3b95a-13ab-453d-a5a5-d9037ebcfada\") " pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-dd67d"
Oct 30 22:04:41.764077 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:41.763947    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3552416b-6da1-4c28-be4b-5450abc40f79-workload-certs\") pod \"e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-pxzmr\" (UID: \"3552416b-6da1-4c28-be4b-5450abc40f79\") " pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-pxzmr"
Oct 30 22:04:41.764077 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:41.763981    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9dc3b95a-13ab-453d-a5a5-d9037ebcfada-credential-socket\") pod \"e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-dd67d\" (UID: \"9dc3b95a-13ab-453d-a5a5-d9037ebcfada\") " pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-dd67d"
Oct 30 22:04:41.764191 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:41.764145    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3552416b-6da1-4c28-be4b-5450abc40f79-istiod-ca-cert\") pod \"e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-pxzmr\" (UID: \"3552416b-6da1-4c28-be4b-5450abc40f79\") " pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-pxzmr"
Oct 30 22:04:41.764247 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:41.764206    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n825w\" (UniqueName: \"kubernetes.io/projected/3552416b-6da1-4c28-be4b-5450abc40f79-kube-api-access-n825w\") pod \"e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-pxzmr\" (UID: \"3552416b-6da1-4c28-be4b-5450abc40f79\") " pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-pxzmr"
Oct 30 22:04:41.764354 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:41.764337    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9dc3b95a-13ab-453d-a5a5-d9037ebcfada-istio-token\") pod \"e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-dd67d\" (UID: \"9dc3b95a-13ab-453d-a5a5-d9037ebcfada\") " pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-dd67d"
Oct 30 22:04:41.764418 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:41.764405    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3552416b-6da1-4c28-be4b-5450abc40f79-credential-socket\") pod \"e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-pxzmr\" (UID: \"3552416b-6da1-4c28-be4b-5450abc40f79\") " pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-pxzmr"
Oct 30 22:04:41.764490 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:41.764435    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9dc3b95a-13ab-453d-a5a5-d9037ebcfada-istio-data\") pod \"e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-dd67d\" (UID: \"9dc3b95a-13ab-453d-a5a5-d9037ebcfada\") " pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-dd67d"
Oct 30 22:04:41.764490 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:41.764472    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9dc3b95a-13ab-453d-a5a5-d9037ebcfada-istiod-ca-cert\") pod \"e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-dd67d\" (UID: \"9dc3b95a-13ab-453d-a5a5-d9037ebcfada\") " pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-dd67d"
Oct 30 22:04:41.764590 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:41.764531    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dxjxj\" (UniqueName: \"kubernetes.io/projected/9dc3b95a-13ab-453d-a5a5-d9037ebcfada-kube-api-access-dxjxj\") pod \"e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-dd67d\" (UID: \"9dc3b95a-13ab-453d-a5a5-d9037ebcfada\") " pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-dd67d"
Oct 30 22:04:41.764590 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:41.764577    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3552416b-6da1-4c28-be4b-5450abc40f79-istio-token\") pod \"e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-pxzmr\" (UID: \"3552416b-6da1-4c28-be4b-5450abc40f79\") " pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-pxzmr"
Oct 30 22:04:41.764691 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:41.764606    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3552416b-6da1-4c28-be4b-5450abc40f79-istio-podinfo\") pod \"e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-pxzmr\" (UID: \"3552416b-6da1-4c28-be4b-5450abc40f79\") " pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-pxzmr"
Oct 30 22:04:41.764742 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:41.764720    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9dc3b95a-13ab-453d-a5a5-d9037ebcfada-istio-podinfo\") pod \"e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-dd67d\" (UID: \"9dc3b95a-13ab-453d-a5a5-d9037ebcfada\") " pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-dd67d"
Oct 30 22:04:41.764795 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:41.764762    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3552416b-6da1-4c28-be4b-5450abc40f79-workload-socket\") pod \"e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-pxzmr\" (UID: \"3552416b-6da1-4c28-be4b-5450abc40f79\") " pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-pxzmr"
Oct 30 22:04:41.764850 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:41.764802    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3552416b-6da1-4c28-be4b-5450abc40f79-istio-envoy\") pod \"e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-pxzmr\" (UID: \"3552416b-6da1-4c28-be4b-5450abc40f79\") " pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-pxzmr"
Oct 30 22:04:41.764850 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:41.764837    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9dc3b95a-13ab-453d-a5a5-d9037ebcfada-workload-socket\") pod \"e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-dd67d\" (UID: \"9dc3b95a-13ab-453d-a5a5-d9037ebcfada\") " pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-dd67d"
Oct 30 22:04:41.765837 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:41.765330    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9dc3b95a-13ab-453d-a5a5-d9037ebcfada-workload-socket\") pod \"e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-dd67d\" (UID: \"9dc3b95a-13ab-453d-a5a5-d9037ebcfada\") " pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-dd67d"
Oct 30 22:04:41.765837 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:41.765347    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9dc3b95a-13ab-453d-a5a5-d9037ebcfada-credential-socket\") pod \"e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-dd67d\" (UID: \"9dc3b95a-13ab-453d-a5a5-d9037ebcfada\") " pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-dd67d"
Oct 30 22:04:41.765837 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:41.765739    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9dc3b95a-13ab-453d-a5a5-d9037ebcfada-istio-data\") pod \"e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-dd67d\" (UID: \"9dc3b95a-13ab-453d-a5a5-d9037ebcfada\") " pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-dd67d"
Oct 30 22:04:41.766087 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:41.766044    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9dc3b95a-13ab-453d-a5a5-d9037ebcfada-istiod-ca-cert\") pod \"e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-dd67d\" (UID: \"9dc3b95a-13ab-453d-a5a5-d9037ebcfada\") " pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-dd67d"
Oct 30 22:04:41.766362 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:41.766341    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9dc3b95a-13ab-453d-a5a5-d9037ebcfada-workload-certs\") pod \"e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-dd67d\" (UID: \"9dc3b95a-13ab-453d-a5a5-d9037ebcfada\") " pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-dd67d"
Oct 30 22:04:41.768053 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:41.768036    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9dc3b95a-13ab-453d-a5a5-d9037ebcfada-istio-envoy\") pod \"e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-dd67d\" (UID: \"9dc3b95a-13ab-453d-a5a5-d9037ebcfada\") " pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-dd67d"
Oct 30 22:04:41.768103 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:41.768051    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9dc3b95a-13ab-453d-a5a5-d9037ebcfada-istio-podinfo\") pod \"e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-dd67d\" (UID: \"9dc3b95a-13ab-453d-a5a5-d9037ebcfada\") " pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-dd67d"
Oct 30 22:04:41.823833 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:41.823801    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9dc3b95a-13ab-453d-a5a5-d9037ebcfada-istio-token\") pod \"e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-dd67d\" (UID: \"9dc3b95a-13ab-453d-a5a5-d9037ebcfada\") " pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-dd67d"
Oct 30 22:04:41.824007 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:41.823952    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dxjxj\" (UniqueName: \"kubernetes.io/projected/9dc3b95a-13ab-453d-a5a5-d9037ebcfada-kube-api-access-dxjxj\") pod \"e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-dd67d\" (UID: \"9dc3b95a-13ab-453d-a5a5-d9037ebcfada\") " pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-dd67d"
Oct 30 22:04:41.866693 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:41.866652    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3552416b-6da1-4c28-be4b-5450abc40f79-credential-socket\") pod \"e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-pxzmr\" (UID: \"3552416b-6da1-4c28-be4b-5450abc40f79\") " pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-pxzmr"
Oct 30 22:04:41.866889 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:41.866740    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3552416b-6da1-4c28-be4b-5450abc40f79-istio-token\") pod \"e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-pxzmr\" (UID: \"3552416b-6da1-4c28-be4b-5450abc40f79\") " pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-pxzmr"
Oct 30 22:04:41.866889 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:41.866765    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3552416b-6da1-4c28-be4b-5450abc40f79-istio-podinfo\") pod \"e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-pxzmr\" (UID: \"3552416b-6da1-4c28-be4b-5450abc40f79\") " pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-pxzmr"
Oct 30 22:04:41.866889 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:41.866858    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3552416b-6da1-4c28-be4b-5450abc40f79-workload-socket\") pod \"e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-pxzmr\" (UID: \"3552416b-6da1-4c28-be4b-5450abc40f79\") " pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-pxzmr"
Oct 30 22:04:41.867113 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:41.866892    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3552416b-6da1-4c28-be4b-5450abc40f79-istio-envoy\") pod \"e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-pxzmr\" (UID: \"3552416b-6da1-4c28-be4b-5450abc40f79\") " pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-pxzmr"
Oct 30 22:04:41.867113 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:41.866996    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3552416b-6da1-4c28-be4b-5450abc40f79-istio-data\") pod \"e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-pxzmr\" (UID: \"3552416b-6da1-4c28-be4b-5450abc40f79\") " pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-pxzmr"
Oct 30 22:04:41.867215 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:41.867130    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3552416b-6da1-4c28-be4b-5450abc40f79-workload-certs\") pod \"e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-pxzmr\" (UID: \"3552416b-6da1-4c28-be4b-5450abc40f79\") " pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-pxzmr"
Oct 30 22:04:41.867215 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:41.867153    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3552416b-6da1-4c28-be4b-5450abc40f79-credential-socket\") pod \"e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-pxzmr\" (UID: \"3552416b-6da1-4c28-be4b-5450abc40f79\") " pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-pxzmr"
Oct 30 22:04:41.867468 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:41.867444    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3552416b-6da1-4c28-be4b-5450abc40f79-workload-socket\") pod \"e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-pxzmr\" (UID: \"3552416b-6da1-4c28-be4b-5450abc40f79\") " pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-pxzmr"
Oct 30 22:04:41.867550 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:41.867467    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3552416b-6da1-4c28-be4b-5450abc40f79-istio-data\") pod \"e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-pxzmr\" (UID: \"3552416b-6da1-4c28-be4b-5450abc40f79\") " pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-pxzmr"
Oct 30 22:04:41.867751 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:41.867705    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3552416b-6da1-4c28-be4b-5450abc40f79-workload-certs\") pod \"e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-pxzmr\" (UID: \"3552416b-6da1-4c28-be4b-5450abc40f79\") " pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-pxzmr"
Oct 30 22:04:41.867751 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:41.867172    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3552416b-6da1-4c28-be4b-5450abc40f79-istiod-ca-cert\") pod \"e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-pxzmr\" (UID: \"3552416b-6da1-4c28-be4b-5450abc40f79\") " pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-pxzmr"
Oct 30 22:04:41.867936 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:41.867777    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3552416b-6da1-4c28-be4b-5450abc40f79-istiod-ca-cert\") pod \"e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-pxzmr\" (UID: \"3552416b-6da1-4c28-be4b-5450abc40f79\") " pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-pxzmr"
Oct 30 22:04:41.867936 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:41.867787    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n825w\" (UniqueName: \"kubernetes.io/projected/3552416b-6da1-4c28-be4b-5450abc40f79-kube-api-access-n825w\") pod \"e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-pxzmr\" (UID: \"3552416b-6da1-4c28-be4b-5450abc40f79\") " pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-pxzmr"
Oct 30 22:04:41.870307 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:41.870280    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3552416b-6da1-4c28-be4b-5450abc40f79-istio-envoy\") pod \"e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-pxzmr\" (UID: \"3552416b-6da1-4c28-be4b-5450abc40f79\") " pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-pxzmr"
Oct 30 22:04:41.870462 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:41.870445    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3552416b-6da1-4c28-be4b-5450abc40f79-istio-podinfo\") pod \"e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-pxzmr\" (UID: \"3552416b-6da1-4c28-be4b-5450abc40f79\") " pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-pxzmr"
Oct 30 22:04:41.924500 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:41.924425    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3552416b-6da1-4c28-be4b-5450abc40f79-istio-token\") pod \"e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-pxzmr\" (UID: \"3552416b-6da1-4c28-be4b-5450abc40f79\") " pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-pxzmr"
Oct 30 22:04:41.924837 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:41.924815    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n825w\" (UniqueName: \"kubernetes.io/projected/3552416b-6da1-4c28-be4b-5450abc40f79-kube-api-access-n825w\") pod \"e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-pxzmr\" (UID: \"3552416b-6da1-4c28-be4b-5450abc40f79\") " pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-pxzmr"
Oct 30 22:04:41.966736 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:41.966707    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-dd67d"
Oct 30 22:04:41.977718 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:41.977690    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-pxzmr"
Oct 30 22:04:42.065771 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:42.065218    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-8cv5d"]
Oct 30 22:04:42.067607 ip-10-0-140-18 kubenswrapper[4473]: W1030 22:04:42.067573    4473 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85b6bb64_5545_4a61_bb0d_a4851348fe30.slice/crio-7c2e7eef5805861fa6e3643983d3dd6be565b5ca7295b921f3f989976b0c8f54 WatchSource:0}: Error finding container 7c2e7eef5805861fa6e3643983d3dd6be565b5ca7295b921f3f989976b0c8f54: Status 404 returned error can't find the container with id 7c2e7eef5805861fa6e3643983d3dd6be565b5ca7295b921f3f989976b0c8f54
Oct 30 22:04:42.070802 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:42.070754    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:04:42.070908 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:42.070831    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:04:42.070908 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:42.070872    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:04:42.355323 ip-10-0-140-18 kubenswrapper[4473]: W1030 22:04:42.355293    4473 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9dc3b95a_13ab_453d_a5a5_d9037ebcfada.slice/crio-110aa388ca17fdd00f4057aafb86e0613da00b464fdd603cbf3702b4d8c70d75 WatchSource:0}: Error finding container 110aa388ca17fdd00f4057aafb86e0613da00b464fdd603cbf3702b4d8c70d75: Status 404 returned error can't find the container with id 110aa388ca17fdd00f4057aafb86e0613da00b464fdd603cbf3702b4d8c70d75
Oct 30 22:04:42.358424 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:42.358382    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:04:42.358522 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:42.358468    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:04:42.358522 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:42.358509    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:04:42.360173 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:42.360150    4473 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4c4ef828-d895-488c-900d-3525e3fda397" path="/var/lib/kubelet/pods/4c4ef828-d895-488c-900d-3525e3fda397/volumes"
Oct 30 22:04:42.360482 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:42.360469    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-dd67d"]
Oct 30 22:04:42.404134 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:42.404099    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-pxzmr"]
Oct 30 22:04:42.404251 ip-10-0-140-18 kubenswrapper[4473]: W1030 22:04:42.404217    4473 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3552416b_6da1_4c28_be4b_5450abc40f79.slice/crio-af7bf2b74bc83da653d00496cb706f93ddec839079216739dcb57aafa2b1ff6a WatchSource:0}: Error finding container af7bf2b74bc83da653d00496cb706f93ddec839079216739dcb57aafa2b1ff6a: Status 404 returned error can't find the container with id af7bf2b74bc83da653d00496cb706f93ddec839079216739dcb57aafa2b1ff6a
Oct 30 22:04:42.407186 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:42.407153    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:04:42.407260 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:42.407224    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:04:42.407311 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:42.407269    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:04:42.885323 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:42.885286    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-pxzmr" event={"ID":"3552416b-6da1-4c28-be4b-5450abc40f79","Type":"ContainerStarted","Data":"42de881e49dd00d4bb57e44377c0ceda3c35c6e6b3cf143c396a68a2d0b4b5ec"}
Oct 30 22:04:42.885966 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:42.885878    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-pxzmr" event={"ID":"3552416b-6da1-4c28-be4b-5450abc40f79","Type":"ContainerStarted","Data":"af7bf2b74bc83da653d00496cb706f93ddec839079216739dcb57aafa2b1ff6a"}
Oct 30 22:04:42.889224 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:42.889201    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-dd67d" event={"ID":"9dc3b95a-13ab-453d-a5a5-d9037ebcfada","Type":"ContainerStarted","Data":"e932b7049a0f917709c2793d151e7e4e8cce450cf4c793d2379ae3c806f88391"}
Oct 30 22:04:42.889403 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:42.889390    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-dd67d" event={"ID":"9dc3b95a-13ab-453d-a5a5-d9037ebcfada","Type":"ContainerStarted","Data":"110aa388ca17fdd00f4057aafb86e0613da00b464fdd603cbf3702b4d8c70d75"}
Oct 30 22:04:42.893157 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:42.893136    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-8cv5d" event={"ID":"85b6bb64-5545-4a61-bb0d-a4851348fe30","Type":"ContainerStarted","Data":"9353c1fa5252a41781bb2c79d223f7f527a1e245af4c585422c54ef6382fa335"}
Oct 30 22:04:42.893319 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:42.893306    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-8cv5d" event={"ID":"85b6bb64-5545-4a61-bb0d-a4851348fe30","Type":"ContainerStarted","Data":"7c2e7eef5805861fa6e3643983d3dd6be565b5ca7295b921f3f989976b0c8f54"}
Oct 30 22:04:42.967275 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:42.967237    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-dd67d"
Oct 30 22:04:42.978878 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:42.978842    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-pxzmr"
Oct 30 22:04:43.019993 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:43.019938    4473 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-pxzmr" podStartSLOduration=2.01990396 podStartE2EDuration="2.01990396s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:04:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:04:43.016733898 +0000 UTC m=+1328.790102726" watchObservedRunningTime="2026-10-30 22:04:43.01990396 +0000 UTC m=+1328.793272446"
Oct 30 22:04:43.149011 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:43.140166    4473 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-8cv5d" podStartSLOduration=2.140143564 podStartE2EDuration="2.140143564s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:04:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:04:43.137443966 +0000 UTC m=+1328.910812453" watchObservedRunningTime="2026-10-30 22:04:43.140143564 +0000 UTC m=+1328.913512052"
Oct 30 22:04:43.271961 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:43.267565    4473 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-dd67d" podStartSLOduration=2.267540114 podStartE2EDuration="2.267540114s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:04:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:04:43.264432194 +0000 UTC m=+1329.037800685" watchObservedRunningTime="2026-10-30 22:04:43.267540114 +0000 UTC m=+1329.040908602"
Oct 30 22:04:43.623088 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:43.623050    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-8cv5d"
Oct 30 22:04:43.624664 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:43.624638    4473 patch_prober.go:29] interesting pod/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-8cv5d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.87:15021/healthz/ready\": dial tcp 10.133.0.87:15021: connect: connection refused" start-of-body=
Oct 30 22:04:43.624777 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:43.624685    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-8cv5d" podUID="85b6bb64-5545-4a61-bb0d-a4851348fe30" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.87:15021/healthz/ready\": dial tcp 10.133.0.87:15021: connect: connection refused"
Oct 30 22:04:43.969003 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:43.968792    4473 patch_prober.go:29] interesting pod/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-dd67d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.88:15021/healthz/ready\": dial tcp 10.133.0.88:15021: connect: connection refused" start-of-body=
Oct 30 22:04:43.969003 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:43.968854    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-dd67d" podUID="9dc3b95a-13ab-453d-a5a5-d9037ebcfada" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.88:15021/healthz/ready\": dial tcp 10.133.0.88:15021: connect: connection refused"
Oct 30 22:04:44.002948 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:43.995150    4473 patch_prober.go:29] interesting pod/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-pxzmr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.89:15021/healthz/ready\": dial tcp 10.133.0.89:15021: connect: connection refused" start-of-body=
Oct 30 22:04:44.002948 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:43.995212    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-pxzmr" podUID="3552416b-6da1-4c28-be4b-5450abc40f79" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.89:15021/healthz/ready\": dial tcp 10.133.0.89:15021: connect: connection refused"
Oct 30 22:04:44.622990 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:44.622947    4473 patch_prober.go:29] interesting pod/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-8cv5d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.87:15021/healthz/ready\": dial tcp 10.133.0.87:15021: connect: connection refused" start-of-body=
Oct 30 22:04:44.623231 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:44.623020    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-8cv5d" podUID="85b6bb64-5545-4a61-bb0d-a4851348fe30" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.87:15021/healthz/ready\": dial tcp 10.133.0.87:15021: connect: connection refused"
Oct 30 22:04:44.997248 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:44.984504    4473 patch_prober.go:29] interesting pod/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-dd67d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.88:15021/healthz/ready\": dial tcp 10.133.0.88:15021: connect: connection refused" start-of-body=
Oct 30 22:04:44.997248 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:44.984562    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-dd67d" podUID="9dc3b95a-13ab-453d-a5a5-d9037ebcfada" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.88:15021/healthz/ready\": dial tcp 10.133.0.88:15021: connect: connection refused"
Oct 30 22:04:45.011219 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:45.011040    4473 patch_prober.go:29] interesting pod/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-pxzmr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.89:15021/healthz/ready\": dial tcp 10.133.0.89:15021: connect: connection refused" start-of-body=
Oct 30 22:04:45.011219 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:45.011110    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-pxzmr" podUID="3552416b-6da1-4c28-be4b-5450abc40f79" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.89:15021/healthz/ready\": dial tcp 10.133.0.89:15021: connect: connection refused"
Oct 30 22:04:45.637944 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:45.629804    4473 patch_prober.go:29] interesting pod/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-8cv5d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.87:15021/healthz/ready\": dial tcp 10.133.0.87:15021: connect: connection refused" start-of-body=
Oct 30 22:04:45.637944 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:45.629869    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-8cv5d" podUID="85b6bb64-5545-4a61-bb0d-a4851348fe30" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.87:15021/healthz/ready\": dial tcp 10.133.0.87:15021: connect: connection refused"
Oct 30 22:04:45.982017 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:45.975353    4473 patch_prober.go:29] interesting pod/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-dd67d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.88:15021/healthz/ready\": dial tcp 10.133.0.88:15021: connect: connection refused" start-of-body=
Oct 30 22:04:45.982017 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:45.976039    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-dd67d" podUID="9dc3b95a-13ab-453d-a5a5-d9037ebcfada" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.88:15021/healthz/ready\": dial tcp 10.133.0.88:15021: connect: connection refused"
Oct 30 22:04:46.009946 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:46.008393    4473 patch_prober.go:29] interesting pod/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-pxzmr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.89:15021/healthz/ready\": dial tcp 10.133.0.89:15021: connect: connection refused" start-of-body=
Oct 30 22:04:46.009946 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:46.008468    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-pxzmr" podUID="3552416b-6da1-4c28-be4b-5450abc40f79" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.89:15021/healthz/ready\": dial tcp 10.133.0.89:15021: connect: connection refused"
Oct 30 22:04:46.623852 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:46.623814    4473 patch_prober.go:29] interesting pod/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-8cv5d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.87:15021/healthz/ready\": dial tcp 10.133.0.87:15021: connect: connection refused" start-of-body=
Oct 30 22:04:46.624063 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:46.623873    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-8cv5d" podUID="85b6bb64-5545-4a61-bb0d-a4851348fe30" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.87:15021/healthz/ready\": dial tcp 10.133.0.87:15021: connect: connection refused"
Oct 30 22:04:46.970012 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:46.969851    4473 patch_prober.go:29] interesting pod/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-dd67d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.88:15021/healthz/ready\": dial tcp 10.133.0.88:15021: connect: connection refused" start-of-body=
Oct 30 22:04:46.970012 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:46.969937    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-dd67d" podUID="9dc3b95a-13ab-453d-a5a5-d9037ebcfada" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.88:15021/healthz/ready\": dial tcp 10.133.0.88:15021: connect: connection refused"
Oct 30 22:04:46.993946 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:46.992894    4473 patch_prober.go:29] interesting pod/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-pxzmr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.89:15021/healthz/ready\": dial tcp 10.133.0.89:15021: connect: connection refused" start-of-body=
Oct 30 22:04:46.993946 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:46.992972    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-pxzmr" podUID="3552416b-6da1-4c28-be4b-5450abc40f79" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.89:15021/healthz/ready\": dial tcp 10.133.0.89:15021: connect: connection refused"
Oct 30 22:04:47.747944 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:47.731622    4473 patch_prober.go:29] interesting pod/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-8cv5d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.87:15021/healthz/ready\": dial tcp 10.133.0.87:15021: connect: connection refused" start-of-body=
Oct 30 22:04:47.747944 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:47.731683    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-8cv5d" podUID="85b6bb64-5545-4a61-bb0d-a4851348fe30" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.87:15021/healthz/ready\": dial tcp 10.133.0.87:15021: connect: connection refused"
Oct 30 22:04:47.976951 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:47.975841    4473 patch_prober.go:29] interesting pod/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-dd67d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.88:15021/healthz/ready\": dial tcp 10.133.0.88:15021: connect: connection refused" start-of-body=
Oct 30 22:04:47.976951 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:47.975905    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-dd67d" podUID="9dc3b95a-13ab-453d-a5a5-d9037ebcfada" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.88:15021/healthz/ready\": dial tcp 10.133.0.88:15021: connect: connection refused"
Oct 30 22:04:48.058346 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:48.056517    4473 patch_prober.go:29] interesting pod/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-pxzmr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.89:15021/healthz/ready\": dial tcp 10.133.0.89:15021: connect: connection refused" start-of-body=
Oct 30 22:04:48.058346 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:48.056582    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-pxzmr" podUID="3552416b-6da1-4c28-be4b-5450abc40f79" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.89:15021/healthz/ready\": dial tcp 10.133.0.89:15021: connect: connection refused"
Oct 30 22:04:48.177957 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:48.170198    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-0f42cb/distinct-2-w3-0f42cb-kserve-fc477897b-jk8qc" podUID="61b49182-057e-41af-b609-1a6abf82c197" containerName="main" probeResult="failure" output="Get \"https://10.133.0.80:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:04:48.280686 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:48.280158    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-9f8aea/distinct-w5-9f8aea-kserve-5b564579cd-kmxzz" podUID="b11e8b11-334f-4a9e-98c2-b0164567a8af" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:04:48.324126 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:48.324023    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-0f42cb/distinct-w3-0f42cb-kserve-fb54fdd59-n6w4l" podUID="ff3c2b25-08aa-4b70-bf57-9cf4a11d05ed" containerName="main" probeResult="failure" output="Get \"https://10.133.0.73:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:04:48.679566 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:48.679528    4473 patch_prober.go:29] interesting pod/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-8cv5d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.87:15021/healthz/ready\": dial tcp 10.133.0.87:15021: connect: connection refused" start-of-body=
Oct 30 22:04:48.778590 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:48.771057    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-8cv5d" podUID="85b6bb64-5545-4a61-bb0d-a4851348fe30" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.87:15021/healthz/ready\": dial tcp 10.133.0.87:15021: connect: connection refused"
Oct 30 22:04:48.778590 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:48.770695    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-29cbad/premium-w6-29cbad-kserve-6c9f8d6899-qtv6c" podUID="381b11f6-3fde-4de7-9ad3-7c0891e76ba1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:04:48.984190 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:48.984066    4473 patch_prober.go:29] interesting pod/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-pxzmr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.89:15021/healthz/ready\": dial tcp 10.133.0.89:15021: connect: connection refused" start-of-body=
Oct 30 22:04:48.984520 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:48.984487    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-pxzmr" podUID="3552416b-6da1-4c28-be4b-5450abc40f79" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.89:15021/healthz/ready\": dial tcp 10.133.0.89:15021: connect: connection refused"
Oct 30 22:04:49.038950 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:49.007998    4473 patch_prober.go:29] interesting pod/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-dd67d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.88:15021/healthz/ready\": dial tcp 10.133.0.88:15021: connect: connection refused" start-of-body=
Oct 30 22:04:49.039343 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:49.039302    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-dd67d" podUID="9dc3b95a-13ab-453d-a5a5-d9037ebcfada" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.88:15021/healthz/ready\": dial tcp 10.133.0.88:15021: connect: connection refused"
Oct 30 22:04:49.137059 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:49.137011    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-dde1e8/distinct-2-w0-dde1e8-kserve-744857d57-ftt8b" podUID="577b020d-2a6b-4020-863c-cf19d4f0ca3b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.79:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:04:49.199382 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:49.138363    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-dde1e8/distinct-w0-dde1e8-kserve-75b68b4c5d-hts2k" podUID="9d72e7c9-657e-4278-8ed9-53adb75bd530" containerName="main" probeResult="failure" output="Get \"https://10.133.0.72:8000/ready\": context deadline exceeded"
Oct 30 22:04:49.199898 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:49.170548    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-0f42cb/distinct-2-w3-0f42cb-kserve-fc477897b-jk8qc" podUID="61b49182-057e-41af-b609-1a6abf82c197" containerName="main" probeResult="failure" output="Get \"https://10.133.0.80:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:04:49.208947 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:49.201292    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-0f42cb/distinct-2-w3-0f42cb-kserve-fc477897b-jk8qc"
Oct 30 22:04:49.344016 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:49.337499    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-0f42cb/distinct-w3-0f42cb-kserve-fb54fdd59-n6w4l" podUID="ff3c2b25-08aa-4b70-bf57-9cf4a11d05ed" containerName="main" probeResult="failure" output="Get \"https://10.133.0.73:8000/ready\": context deadline exceeded"
Oct 30 22:04:49.352765 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:49.352339    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-0f42cb/distinct-w3-0f42cb-kserve-fb54fdd59-n6w4l"
Oct 30 22:04:49.360954 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:49.353290    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-9f8aea/distinct-w5-9f8aea-kserve-5b564579cd-kmxzz" podUID="b11e8b11-334f-4a9e-98c2-b0164567a8af" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": context deadline exceeded"
Oct 30 22:04:49.377990 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:49.377937    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-0c99b1/distinct-2-w2-0c99b1-kserve-7466864658-8z7fk" podUID="89c09555-cf53-49a7-9c3b-95d2da2a293a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.82:8000/ready\": context deadline exceeded"
Oct 30 22:04:49.401986 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:49.380341    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-9f8aea/distinct-w5-9f8aea-kserve-5b564579cd-kmxzz"
Oct 30 22:04:49.642952 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:49.642897    4473 patch_prober.go:29] interesting pod/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-8cv5d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.87:15021/healthz/ready\": dial tcp 10.133.0.87:15021: connect: connection refused" start-of-body=
Oct 30 22:04:49.643269 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:49.643226    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-8cv5d" podUID="85b6bb64-5545-4a61-bb0d-a4851348fe30" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.87:15021/healthz/ready\": dial tcp 10.133.0.87:15021: connect: connection refused"
Oct 30 22:04:49.824021 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:49.823968    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-29cbad/premium-w6-29cbad-kserve-6c9f8d6899-qtv6c" podUID="381b11f6-3fde-4de7-9ad3-7c0891e76ba1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.60:8000/ready\": context deadline exceeded"
Oct 30 22:04:50.085577 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:50.085242    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-29cbad/premium-w6-29cbad-kserve-6c9f8d6899-qtv6c"
Oct 30 22:04:50.089947 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:49.991117    4473 patch_prober.go:29] interesting pod/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-dd67d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.88:15021/healthz/ready\": dial tcp 10.133.0.88:15021: connect: connection refused" start-of-body=
Oct 30 22:04:50.089947 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:50.086194    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-dd67d" podUID="9dc3b95a-13ab-453d-a5a5-d9037ebcfada" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.88:15021/healthz/ready\": dial tcp 10.133.0.88:15021: connect: connection refused"
Oct 30 22:04:50.089947 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:50.028513    4473 patch_prober.go:29] interesting pod/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-pxzmr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.89:15021/healthz/ready\": dial tcp 10.133.0.89:15021: connect: connection refused" start-of-body=
Oct 30 22:04:50.089947 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:50.086251    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-pxzmr" podUID="3552416b-6da1-4c28-be4b-5450abc40f79" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.89:15021/healthz/ready\": dial tcp 10.133.0.89:15021: connect: connection refused"
Oct 30 22:04:50.198112 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:50.198057    4473 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w6-29cbad/premium-w6-29cbad-kserve-6c9f8d6899-qtv6c" podUID="381b11f6-3fde-4de7-9ad3-7c0891e76ba1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.60:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:04:50.201086 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:50.201023    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-dde1e8/distinct-w0-dde1e8-kserve-75b68b4c5d-hts2k" podUID="9d72e7c9-657e-4278-8ed9-53adb75bd530" containerName="main" probeResult="failure" output="Get \"https://10.133.0.72:8000/ready\": context deadline exceeded"
Oct 30 22:04:50.202730 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:50.202697    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-dde1e8/distinct-2-w0-dde1e8-kserve-744857d57-ftt8b" podUID="577b020d-2a6b-4020-863c-cf19d4f0ca3b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.79:8000/ready\": context deadline exceeded"
Oct 30 22:04:50.207094 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:50.207067    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-dde1e8/distinct-w0-dde1e8-kserve-75b68b4c5d-hts2k"
Oct 30 22:04:50.207328 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:50.207312    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-dde1e8/distinct-2-w0-dde1e8-kserve-744857d57-ftt8b"
Oct 30 22:04:50.218143 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:50.218099    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-0f42cb/distinct-2-w3-0f42cb-kserve-fc477897b-jk8qc" podUID="61b49182-057e-41af-b609-1a6abf82c197" containerName="main" probeResult="failure" output="Get \"https://10.133.0.80:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:04:50.423119 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:50.390392    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-0c99b1/distinct-2-w2-0c99b1-kserve-7466864658-8z7fk" podUID="89c09555-cf53-49a7-9c3b-95d2da2a293a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.82:8000/ready\": context deadline exceeded"
Oct 30 22:04:50.515948 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:50.407262    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-0f42cb/distinct-w3-0f42cb-kserve-fb54fdd59-n6w4l" podUID="ff3c2b25-08aa-4b70-bf57-9cf4a11d05ed" containerName="main" probeResult="failure" output="Get \"https://10.133.0.73:8000/ready\": context deadline exceeded"
Oct 30 22:04:51.067647 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:50.407307    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-9f8aea/distinct-w5-9f8aea-kserve-5b564579cd-kmxzz" podUID="b11e8b11-334f-4a9e-98c2-b0164567a8af" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": context deadline exceeded"
Oct 30 22:04:51.076129 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:50.614456    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-0c99b1/distinct-2-w2-0c99b1-kserve-7466864658-8z7fk"
Oct 30 22:04:51.077640 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:50.629290    4473 patch_prober.go:29] interesting pod/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-8cv5d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.87:15021/healthz/ready\": dial tcp 10.133.0.87:15021: connect: connection refused" start-of-body=
Oct 30 22:04:51.077866 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:51.077828    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-8cv5d" podUID="85b6bb64-5545-4a61-bb0d-a4851348fe30" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.87:15021/healthz/ready\": dial tcp 10.133.0.87:15021: connect: connection refused"
Oct 30 22:04:51.078049 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:04:50.878358    4473 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fncvg" containerName="extractor"
Oct 30 22:04:51.078194 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:50.975135    4473 patch_prober.go:29] interesting pod/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-dd67d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.88:15021/healthz/ready\": dial tcp 10.133.0.88:15021: connect: connection refused" start-of-body=
Oct 30 22:04:51.078344 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:51.078323    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-dd67d" podUID="9dc3b95a-13ab-453d-a5a5-d9037ebcfada" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.88:15021/healthz/ready\": dial tcp 10.133.0.88:15021: connect: connection refused"
Oct 30 22:04:51.078493 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:51.052032    4473 patch_prober.go:29] interesting pod/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-pxzmr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.89:15021/healthz/ready\": dial tcp 10.133.0.89:15021: connect: connection refused" start-of-body=
Oct 30 22:04:51.078632 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:51.078614    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-pxzmr" podUID="3552416b-6da1-4c28-be4b-5450abc40f79" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.89:15021/healthz/ready\": dial tcp 10.133.0.89:15021: connect: connection refused"
Oct 30 22:04:51.097865 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:51.097825    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-29cbad/premium-w6-29cbad-kserve-6c9f8d6899-qtv6c" podUID="381b11f6-3fde-4de7-9ad3-7c0891e76ba1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:04:51.261286 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:51.261235    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-0f42cb/distinct-2-w3-0f42cb-kserve-fc477897b-jk8qc" podUID="61b49182-057e-41af-b609-1a6abf82c197" containerName="main" probeResult="failure" output="Get \"https://10.133.0.80:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:04:51.377216 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:51.377074    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-jzz7s"]
Oct 30 22:04:51.428022 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:51.423993    4473 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-jzz7s" podUID="c9455dd0-779f-4bcb-aa4f-adb7bd780159" containerName="istio-proxy" containerID="cri-o://54e8913e351e83423e8649898d06ddc6e926c776babe72c9ad1ef1e9ef213ad8" gracePeriod=30
Oct 30 22:04:51.509959 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:51.473328    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-0c99b1/distinct-2-w2-0c99b1-kserve-7466864658-8z7fk" podUID="89c09555-cf53-49a7-9c3b-95d2da2a293a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.82:8000/ready\": context deadline exceeded"
Oct 30 22:04:51.828994 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:51.816036    4473 patch_prober.go:29] interesting pod/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-j5tqr container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 22:04:51.829327 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:51.829271    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-dde1e8/distinct-2-w0-dde1e8-kserve-744857d57-ftt8b" podUID="577b020d-2a6b-4020-863c-cf19d4f0ca3b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.79:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:04:51.929259 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:51.929219    4473 patch_prober.go:29] interesting pod/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-8cv5d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.87:15021/healthz/ready\": dial tcp 10.133.0.87:15021: connect: connection refused" start-of-body=
Oct 30 22:04:51.995951 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:51.992560    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-dde1e8/distinct-w0-dde1e8-kserve-75b68b4c5d-hts2k" podUID="9d72e7c9-657e-4278-8ed9-53adb75bd530" containerName="main" probeResult="failure" output="Get \"https://10.133.0.72:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:04:52.169298 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:52.169256    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-9f8aea/distinct-w5-9f8aea-kserve-5b564579cd-kmxzz" podUID="b11e8b11-334f-4a9e-98c2-b0164567a8af" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": context deadline exceeded"
Oct 30 22:04:52.271948 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:52.190865    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-29cbad/premium-w6-29cbad-kserve-6c9f8d6899-qtv6c" podUID="381b11f6-3fde-4de7-9ad3-7c0891e76ba1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:04:52.271948 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:52.190901    4473 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-0f42cb/distinct-2-w3-0f42cb-kserve-fc477897b-jk8qc" podUID="61b49182-057e-41af-b609-1a6abf82c197" containerName="main" probeResult="failure" output="Get \"https://10.133.0.80:8000/health\": context deadline exceeded"
Oct 30 22:04:52.273184 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:52.273136    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-0f42cb/distinct-2-w3-0f42cb-kserve-fc477897b-jk8qc" podUID="61b49182-057e-41af-b609-1a6abf82c197" containerName="main" probeResult="failure" output="Get \"https://10.133.0.80:8000/ready\": context deadline exceeded"
Oct 30 22:04:52.309950 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:52.285326    4473 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-0c99b1/distinct-2-w2-0c99b1-kserve-7466864658-8z7fk" podUID="89c09555-cf53-49a7-9c3b-95d2da2a293a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.82:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:04:52.696840 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:52.344198    4473 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-0f42cb/distinct-w3-0f42cb-kserve-fb54fdd59-n6w4l" podUID="ff3c2b25-08aa-4b70-bf57-9cf4a11d05ed" containerName="main" probeResult="failure" output="Get \"https://10.133.0.73:8000/health\": context deadline exceeded"
Oct 30 22:04:52.764332 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:52.467367    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-j5tqr" podUID="0f0fbc43-d98b-4b7c-872b-7978f76650ef" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:04:52.764595 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:52.467706    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-8cv5d" podUID="85b6bb64-5545-4a61-bb0d-a4851348fe30" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.87:15021/healthz/ready\": dial tcp 10.133.0.87:15021: connect: connection refused"
Oct 30 22:04:52.765040 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:52.509445    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-0f42cb/distinct-w3-0f42cb-kserve-fb54fdd59-n6w4l" podUID="ff3c2b25-08aa-4b70-bf57-9cf4a11d05ed" containerName="main" probeResult="failure" output="Get \"https://10.133.0.73:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:04:52.765486 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:52.512061    4473 patch_prober.go:29] interesting pod/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-dd67d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.88:15021/healthz/ready\": dial tcp 10.133.0.88:15021: connect: connection refused" start-of-body=
Oct 30 22:04:52.765653 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:52.765623    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-dd67d" podUID="9dc3b95a-13ab-453d-a5a5-d9037ebcfada" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.88:15021/healthz/ready\": dial tcp 10.133.0.88:15021: connect: connection refused"
Oct 30 22:04:52.765758 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:52.512248    4473 patch_prober.go:29] interesting pod/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-pxzmr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.89:15021/healthz/ready\": dial tcp 10.133.0.89:15021: connect: connection refused" start-of-body=
Oct 30 22:04:52.765870 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:52.765853    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-pxzmr" podUID="3552416b-6da1-4c28-be4b-5450abc40f79" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.89:15021/healthz/ready\": dial tcp 10.133.0.89:15021: connect: connection refused"
Oct 30 22:04:52.766001 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:52.520006    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-0c99b1/distinct-2-w2-0c99b1-kserve-7466864658-8z7fk" podUID="89c09555-cf53-49a7-9c3b-95d2da2a293a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.82:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:04:52.767205 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:52.767181    4473 patch_prober.go:29] interesting pod/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-8cv5d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.87:15021/healthz/ready\": dial tcp 10.133.0.87:15021: connect: connection refused" start-of-body=
Oct 30 22:04:52.767366 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:52.767341    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-8cv5d" podUID="85b6bb64-5545-4a61-bb0d-a4851348fe30" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.87:15021/healthz/ready\": dial tcp 10.133.0.87:15021: connect: connection refused"
Oct 30 22:04:52.859623 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:52.859088    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-dde1e8/distinct-2-w0-dde1e8-kserve-744857d57-ftt8b" podUID="577b020d-2a6b-4020-863c-cf19d4f0ca3b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.79:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:04:52.993372 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:52.992820    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-dde1e8/distinct-w0-dde1e8-kserve-75b68b4c5d-hts2k" podUID="9d72e7c9-657e-4278-8ed9-53adb75bd530" containerName="main" probeResult="failure" output="Get \"https://10.133.0.72:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:04:53.002612 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:53.002579    4473 patch_prober.go:29] interesting pod/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-dd67d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.88:15021/healthz/ready\": dial tcp 10.133.0.88:15021: connect: connection refused" start-of-body=
Oct 30 22:04:53.016793 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:53.016736    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-dd67d" podUID="9dc3b95a-13ab-453d-a5a5-d9037ebcfada" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.88:15021/healthz/ready\": dial tcp 10.133.0.88:15021: connect: connection refused"
Oct 30 22:04:53.017091 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:53.015112    4473 patch_prober.go:29] interesting pod/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-pxzmr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.89:15021/healthz/ready\": dial tcp 10.133.0.89:15021: connect: connection refused" start-of-body=
Oct 30 22:04:53.017279 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:53.017255    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-pxzmr" podUID="3552416b-6da1-4c28-be4b-5450abc40f79" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.89:15021/healthz/ready\": dial tcp 10.133.0.89:15021: connect: connection refused"
Oct 30 22:04:53.260104 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:53.260009    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-9f8aea/distinct-w5-9f8aea-kserve-5b564579cd-kmxzz" podUID="b11e8b11-334f-4a9e-98c2-b0164567a8af" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:04:53.355085 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:53.328332    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-29cbad/premium-w6-29cbad-kserve-6c9f8d6899-qtv6c" podUID="381b11f6-3fde-4de7-9ad3-7c0891e76ba1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.60:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:04:53.355085 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:53.345286    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-0f42cb/distinct-2-w3-0f42cb-kserve-fc477897b-jk8qc" podUID="61b49182-057e-41af-b609-1a6abf82c197" containerName="main" probeResult="failure" output="Get \"https://10.133.0.80:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:04:53.633727 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:53.633692    4473 patch_prober.go:29] interesting pod/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-8cv5d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.87:15021/healthz/ready\": dial tcp 10.133.0.87:15021: connect: connection refused" start-of-body=
Oct 30 22:04:53.634157 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:53.634022    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-8cv5d" podUID="85b6bb64-5545-4a61-bb0d-a4851348fe30" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.87:15021/healthz/ready\": dial tcp 10.133.0.87:15021: connect: connection refused"
Oct 30 22:04:53.788476 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:53.788414    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-0f42cb/distinct-w3-0f42cb-kserve-fb54fdd59-n6w4l" podUID="ff3c2b25-08aa-4b70-bf57-9cf4a11d05ed" containerName="main" probeResult="failure" output="Get \"https://10.133.0.73:8000/ready\": context deadline exceeded"
Oct 30 22:04:53.820995 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:53.815549    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-0c99b1/distinct-2-w2-0c99b1-kserve-7466864658-8z7fk" podUID="89c09555-cf53-49a7-9c3b-95d2da2a293a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.82:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:04:53.861075 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:53.861011    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-dde1e8/distinct-2-w0-dde1e8-kserve-744857d57-ftt8b" podUID="577b020d-2a6b-4020-863c-cf19d4f0ca3b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.79:8000/ready\": context deadline exceeded"
Oct 30 22:04:53.879577 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:53.879528    4473 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-dde1e8/distinct-w0-dde1e8-kserve-75b68b4c5d-hts2k" podUID="9d72e7c9-657e-4278-8ed9-53adb75bd530" containerName="main" probeResult="failure" output="Get \"https://10.133.0.72:8000/health\": context deadline exceeded"
Oct 30 22:04:54.008011 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:53.990384    4473 patch_prober.go:29] interesting pod/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-dd67d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.88:15021/healthz/ready\": dial tcp 10.133.0.88:15021: connect: connection refused" start-of-body=
Oct 30 22:04:54.008011 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:53.990444    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-dd67d" podUID="9dc3b95a-13ab-453d-a5a5-d9037ebcfada" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.88:15021/healthz/ready\": dial tcp 10.133.0.88:15021: connect: connection refused"
Oct 30 22:04:54.008011 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:53.991546    4473 patch_prober.go:29] interesting pod/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-pxzmr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.89:15021/healthz/ready\": dial tcp 10.133.0.89:15021: connect: connection refused" start-of-body=
Oct 30 22:04:54.008011 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:53.991593    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-pxzmr" podUID="3552416b-6da1-4c28-be4b-5450abc40f79" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.89:15021/healthz/ready\": dial tcp 10.133.0.89:15021: connect: connection refused"
Oct 30 22:04:54.008011 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:53.993954    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-dde1e8/distinct-w0-dde1e8-kserve-75b68b4c5d-hts2k" podUID="9d72e7c9-657e-4278-8ed9-53adb75bd530" containerName="main" probeResult="failure" output="Get \"https://10.133.0.72:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:04:54.019948 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:54.018578    4473 patch_prober.go:29] interesting pod/e2e-worker-w2-0c99b1-openshift-default-967855977-ktzwz container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.86:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 22:04:54.019948 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:54.018643    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-ktzwz" podUID="df9bbdb1-5a45-4223-b03a-5d5e11542e60" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.86:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:04:54.351009 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:54.350344    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-0f42cb/distinct-2-w3-0f42cb-kserve-fc477897b-jk8qc" podUID="61b49182-057e-41af-b609-1a6abf82c197" containerName="main" probeResult="failure" output="Get \"https://10.133.0.80:8000/ready\": context deadline exceeded"
Oct 30 22:04:54.356049 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:54.356006    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-9f8aea/distinct-w5-9f8aea-kserve-5b564579cd-kmxzz" podUID="b11e8b11-334f-4a9e-98c2-b0164567a8af" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": context deadline exceeded"
Oct 30 22:04:54.418090 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:54.396891    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-29cbad/premium-w6-29cbad-kserve-6c9f8d6899-qtv6c" podUID="381b11f6-3fde-4de7-9ad3-7c0891e76ba1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.60:8000/ready\": context deadline exceeded"
Oct 30 22:04:54.494612 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:54.464071    4473 patch_prober.go:29] interesting pod/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-bc2x8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.83:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 22:04:54.494612 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:54.464140    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-bc2x8" podUID="365aaba7-1b9d-463f-9231-b0edad7e1aff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.83:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:04:54.627649 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:54.626552    4473 patch_prober.go:29] interesting pod/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-8cv5d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.87:15021/healthz/ready\": dial tcp 10.133.0.87:15021: connect: connection refused" start-of-body=
Oct 30 22:04:54.627649 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:54.626618    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-8cv5d" podUID="85b6bb64-5545-4a61-bb0d-a4851348fe30" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.87:15021/healthz/ready\": dial tcp 10.133.0.87:15021: connect: connection refused"
Oct 30 22:04:54.790194 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:54.790095    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-0f42cb/distinct-w3-0f42cb-kserve-fb54fdd59-n6w4l" podUID="ff3c2b25-08aa-4b70-bf57-9cf4a11d05ed" containerName="main" probeResult="failure" output="Get \"https://10.133.0.73:8000/ready\": context deadline exceeded"
Oct 30 22:04:54.819016 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:54.818969    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-0c99b1/distinct-2-w2-0c99b1-kserve-7466864658-8z7fk" podUID="89c09555-cf53-49a7-9c3b-95d2da2a293a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.82:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:04:54.874088 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:54.874034    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-dde1e8/distinct-2-w0-dde1e8-kserve-744857d57-ftt8b" podUID="577b020d-2a6b-4020-863c-cf19d4f0ca3b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.79:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:04:54.989442 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:54.989345    4473 patch_prober.go:29] interesting pod/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-dd67d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.88:15021/healthz/ready\": dial tcp 10.133.0.88:15021: connect: connection refused" start-of-body=
Oct 30 22:04:54.990774 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:54.990745    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-dd67d" podUID="9dc3b95a-13ab-453d-a5a5-d9037ebcfada" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.88:15021/healthz/ready\": dial tcp 10.133.0.88:15021: connect: connection refused"
Oct 30 22:04:54.991022 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:54.990669    4473 patch_prober.go:29] interesting pod/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-pxzmr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.89:15021/healthz/ready\": dial tcp 10.133.0.89:15021: connect: connection refused" start-of-body=
Oct 30 22:04:54.991178 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:54.991158    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-pxzmr" podUID="3552416b-6da1-4c28-be4b-5450abc40f79" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.89:15021/healthz/ready\": dial tcp 10.133.0.89:15021: connect: connection refused"
Oct 30 22:04:55.039949 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:55.039014    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-dde1e8/distinct-w0-dde1e8-kserve-75b68b4c5d-hts2k" podUID="9d72e7c9-657e-4278-8ed9-53adb75bd530" containerName="main" probeResult="failure" output="Get \"https://10.133.0.72:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:04:55.233445 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:55.233404    4473 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-dde1e8/distinct-2-w0-dde1e8-kserve-744857d57-ftt8b" podUID="577b020d-2a6b-4020-863c-cf19d4f0ca3b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.79:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:04:55.377086 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:55.375009    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-0f42cb/distinct-2-w3-0f42cb-kserve-fc477897b-jk8qc" podUID="61b49182-057e-41af-b609-1a6abf82c197" containerName="main" probeResult="failure" output="Get \"https://10.133.0.80:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:04:55.407051 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:55.406999    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-9f8aea/distinct-w5-9f8aea-kserve-5b564579cd-kmxzz" podUID="b11e8b11-334f-4a9e-98c2-b0164567a8af" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": context deadline exceeded"
Oct 30 22:04:55.497944 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:55.494392    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-29cbad/premium-w6-29cbad-kserve-6c9f8d6899-qtv6c" podUID="381b11f6-3fde-4de7-9ad3-7c0891e76ba1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.60:8000/ready\": context deadline exceeded"
Oct 30 22:04:55.698072 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:55.697853    4473 patch_prober.go:29] interesting pod/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-8cv5d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.87:15021/healthz/ready\": dial tcp 10.133.0.87:15021: connect: connection refused" start-of-body=
Oct 30 22:04:55.698072 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:55.697914    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-8cv5d" podUID="85b6bb64-5545-4a61-bb0d-a4851348fe30" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.87:15021/healthz/ready\": dial tcp 10.133.0.87:15021: connect: connection refused"
Oct 30 22:04:55.791777 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:55.791736    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-0f42cb/distinct-w3-0f42cb-kserve-fb54fdd59-n6w4l" podUID="ff3c2b25-08aa-4b70-bf57-9cf4a11d05ed" containerName="main" probeResult="failure" output="Get \"https://10.133.0.73:8000/ready\": context deadline exceeded"
Oct 30 22:04:55.941048 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:55.940052    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-dde1e8/distinct-2-w0-dde1e8-kserve-744857d57-ftt8b" podUID="577b020d-2a6b-4020-863c-cf19d4f0ca3b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.79:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:04:55.983375 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:55.980378    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-0c99b1/distinct-2-w2-0c99b1-kserve-7466864658-8z7fk" podUID="89c09555-cf53-49a7-9c3b-95d2da2a293a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.82:8000/ready\": context deadline exceeded"
Oct 30 22:04:56.006942 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:55.981352    4473 patch_prober.go:29] interesting pod/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-dd67d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.88:15021/healthz/ready\": dial tcp 10.133.0.88:15021: connect: connection refused" start-of-body=
Oct 30 22:04:56.006942 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:55.983905    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-dd67d" podUID="9dc3b95a-13ab-453d-a5a5-d9037ebcfada" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.88:15021/healthz/ready\": dial tcp 10.133.0.88:15021: connect: connection refused"
Oct 30 22:04:56.009299 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:56.009267    4473 patch_prober.go:29] interesting pod/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-pxzmr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.89:15021/healthz/ready\": dial tcp 10.133.0.89:15021: connect: connection refused" start-of-body=
Oct 30 22:04:56.019085 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:56.019035    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-pxzmr" podUID="3552416b-6da1-4c28-be4b-5450abc40f79" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.89:15021/healthz/ready\": dial tcp 10.133.0.89:15021: connect: connection refused"
Oct 30 22:04:56.066983 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:56.040186    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-dde1e8/distinct-w0-dde1e8-kserve-75b68b4c5d-hts2k" podUID="9d72e7c9-657e-4278-8ed9-53adb75bd530" containerName="main" probeResult="failure" output="Get \"https://10.133.0.72:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:04:56.105891 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:56.105839    4473 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w5-9f8aea/distinct-w5-9f8aea-kserve-5b564579cd-kmxzz" podUID="b11e8b11-334f-4a9e-98c2-b0164567a8af" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:04:56.407666 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:56.407607    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-0f42cb/distinct-2-w3-0f42cb-kserve-fc477897b-jk8qc" podUID="61b49182-057e-41af-b609-1a6abf82c197" containerName="main" probeResult="failure" output="Get \"https://10.133.0.80:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:04:56.432790 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:56.431622    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-9f8aea/distinct-w5-9f8aea-kserve-5b564579cd-kmxzz" podUID="b11e8b11-334f-4a9e-98c2-b0164567a8af" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": context deadline exceeded"
Oct 30 22:04:56.538141 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:56.538073    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-29cbad/premium-w6-29cbad-kserve-6c9f8d6899-qtv6c" podUID="381b11f6-3fde-4de7-9ad3-7c0891e76ba1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:04:56.640291 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:56.629039    4473 patch_prober.go:29] interesting pod/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-8cv5d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.87:15021/healthz/ready\": dial tcp 10.133.0.87:15021: connect: connection refused" start-of-body=
Oct 30 22:04:56.640658 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:56.640626    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-8cv5d" podUID="85b6bb64-5545-4a61-bb0d-a4851348fe30" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.87:15021/healthz/ready\": dial tcp 10.133.0.87:15021: connect: connection refused"
Oct 30 22:04:56.963437 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:56.956720    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-0f42cb/distinct-w3-0f42cb-kserve-fb54fdd59-n6w4l" podUID="ff3c2b25-08aa-4b70-bf57-9cf4a11d05ed" containerName="main" probeResult="failure" output="Get \"https://10.133.0.73:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:04:57.097115 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:57.034584    4473 patch_prober.go:29] interesting pod/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-pxzmr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.89:15021/healthz/ready\": dial tcp 10.133.0.89:15021: connect: connection refused" start-of-body=
Oct 30 22:04:57.097115 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:57.034655    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-pxzmr" podUID="3552416b-6da1-4c28-be4b-5450abc40f79" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.89:15021/healthz/ready\": dial tcp 10.133.0.89:15021: connect: connection refused"
Oct 30 22:04:57.097115 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:57.062547    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-dde1e8/distinct-2-w0-dde1e8-kserve-744857d57-ftt8b" podUID="577b020d-2a6b-4020-863c-cf19d4f0ca3b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.79:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:04:57.097115 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:57.065729    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-0c99b1/distinct-2-w2-0c99b1-kserve-7466864658-8z7fk" podUID="89c09555-cf53-49a7-9c3b-95d2da2a293a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.82:8000/ready\": context deadline exceeded"
Oct 30 22:04:57.365949 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:57.340785    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-dde1e8/distinct-w0-dde1e8-kserve-75b68b4c5d-hts2k" podUID="9d72e7c9-657e-4278-8ed9-53adb75bd530" containerName="main" probeResult="failure" output="Get \"https://10.133.0.72:8000/ready\": context deadline exceeded"
Oct 30 22:04:57.547592 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:04:57.547484    4473 kubelet.go:2711] "Housekeeping took longer than expected" err="housekeeping took too long" expected="1s" actual="1.175s"
Oct 30 22:04:57.548984 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:57.548962    4473 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-l8zrp"]
Oct 30 22:04:57.631654 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:57.631594    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-29cbad/premium-w6-29cbad-kserve-6c9f8d6899-qtv6c" podUID="381b11f6-3fde-4de7-9ad3-7c0891e76ba1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:04:57.636122 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:57.633485    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-0f42cb/distinct-2-w3-0f42cb-kserve-fc477897b-jk8qc" podUID="61b49182-057e-41af-b609-1a6abf82c197" containerName="main" probeResult="failure" output="Get \"https://10.133.0.80:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:04:57.637874 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:57.637742    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-9f8aea/distinct-w5-9f8aea-kserve-5b564579cd-kmxzz" podUID="b11e8b11-334f-4a9e-98c2-b0164567a8af" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:04:57.649189 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:57.648353    4473 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-jzz7s container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body=
Oct 30 22:04:57.649601 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:57.649542    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-jzz7s" podUID="c9455dd0-779f-4bcb-aa4f-adb7bd780159" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused"
Oct 30 22:04:57.658178 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:57.658141    4473 patch_prober.go:29] interesting pod/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-dd67d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.88:15021/healthz/ready\": dial tcp 10.133.0.88:15021: connect: connection refused" start-of-body=
Oct 30 22:04:57.658475 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:57.658441    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-dd67d" podUID="9dc3b95a-13ab-453d-a5a5-d9037ebcfada" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.88:15021/healthz/ready\": dial tcp 10.133.0.88:15021: connect: connection refused"
Oct 30 22:04:58.022571 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:58.022514    4473 patch_prober.go:29] interesting pod/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-pxzmr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.89:15021/healthz/ready\": dial tcp 10.133.0.89:15021: connect: connection refused" start-of-body=
Oct 30 22:04:58.028065 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:58.022864    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-pxzmr" podUID="3552416b-6da1-4c28-be4b-5450abc40f79" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.89:15021/healthz/ready\": dial tcp 10.133.0.89:15021: connect: connection refused"
Oct 30 22:04:58.085282 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:58.085228    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-0f42cb/distinct-w3-0f42cb-kserve-fb54fdd59-n6w4l" podUID="ff3c2b25-08aa-4b70-bf57-9cf4a11d05ed" containerName="main" probeResult="failure" output="Get \"https://10.133.0.73:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:04:58.101476 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:58.101430    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-dde1e8/distinct-2-w0-dde1e8-kserve-744857d57-ftt8b" podUID="577b020d-2a6b-4020-863c-cf19d4f0ca3b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.79:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:04:58.189943 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:58.165205    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-l8zrp"]
Oct 30 22:04:58.189943 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:58.165266    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-8cv5d"
Oct 30 22:04:58.189943 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:58.165294    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-pxzmr"
Oct 30 22:04:58.189943 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:58.165318    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-dd67d"
Oct 30 22:04:58.189943 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:58.165376    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-0c99b1/distinct-2-w2-0c99b1-kserve-7466864658-8z7fk"
Oct 30 22:04:58.189943 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:58.166479    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-l8zrp"
Oct 30 22:04:58.251375 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:58.251213    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-0f42cb/distinct-2-w3-0f42cb-kserve-fc477897b-jk8qc"
Oct 30 22:04:58.276292 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:58.275480    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-dde1e8/distinct-w0-dde1e8-kserve-75b68b4c5d-hts2k"
Oct 30 22:04:58.291024 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:58.290973    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/78f06b05-32bf-46bc-bd95-2b3a8da72b5b-credential-socket\") pod \"e2e-worker-w3-0f42cb-openshift-default-584f4c4865-l8zrp\" (UID: \"78f06b05-32bf-46bc-bd95-2b3a8da72b5b\") " pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-l8zrp"
Oct 30 22:04:58.291355 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:58.291334    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/78f06b05-32bf-46bc-bd95-2b3a8da72b5b-istio-data\") pod \"e2e-worker-w3-0f42cb-openshift-default-584f4c4865-l8zrp\" (UID: \"78f06b05-32bf-46bc-bd95-2b3a8da72b5b\") " pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-l8zrp"
Oct 30 22:04:58.291537 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:58.291522    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/78f06b05-32bf-46bc-bd95-2b3a8da72b5b-istio-envoy\") pod \"e2e-worker-w3-0f42cb-openshift-default-584f4c4865-l8zrp\" (UID: \"78f06b05-32bf-46bc-bd95-2b3a8da72b5b\") " pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-l8zrp"
Oct 30 22:04:58.291814 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:58.291792    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/78f06b05-32bf-46bc-bd95-2b3a8da72b5b-istiod-ca-cert\") pod \"e2e-worker-w3-0f42cb-openshift-default-584f4c4865-l8zrp\" (UID: \"78f06b05-32bf-46bc-bd95-2b3a8da72b5b\") " pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-l8zrp"
Oct 30 22:04:58.292030 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:58.292008    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/78f06b05-32bf-46bc-bd95-2b3a8da72b5b-istio-podinfo\") pod \"e2e-worker-w3-0f42cb-openshift-default-584f4c4865-l8zrp\" (UID: \"78f06b05-32bf-46bc-bd95-2b3a8da72b5b\") " pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-l8zrp"
Oct 30 22:04:58.292161 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:58.292147    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/78f06b05-32bf-46bc-bd95-2b3a8da72b5b-workload-certs\") pod \"e2e-worker-w3-0f42cb-openshift-default-584f4c4865-l8zrp\" (UID: \"78f06b05-32bf-46bc-bd95-2b3a8da72b5b\") " pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-l8zrp"
Oct 30 22:04:58.292288 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:58.292274    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/78f06b05-32bf-46bc-bd95-2b3a8da72b5b-workload-socket\") pod \"e2e-worker-w3-0f42cb-openshift-default-584f4c4865-l8zrp\" (UID: \"78f06b05-32bf-46bc-bd95-2b3a8da72b5b\") " pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-l8zrp"
Oct 30 22:04:58.292416 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:58.292402    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/78f06b05-32bf-46bc-bd95-2b3a8da72b5b-istio-token\") pod \"e2e-worker-w3-0f42cb-openshift-default-584f4c4865-l8zrp\" (UID: \"78f06b05-32bf-46bc-bd95-2b3a8da72b5b\") " pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-l8zrp"
Oct 30 22:04:58.292578 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:58.292555    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jcfzn\" (UniqueName: \"kubernetes.io/projected/78f06b05-32bf-46bc-bd95-2b3a8da72b5b-kube-api-access-jcfzn\") pod \"e2e-worker-w3-0f42cb-openshift-default-584f4c4865-l8zrp\" (UID: \"78f06b05-32bf-46bc-bd95-2b3a8da72b5b\") " pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-l8zrp"
Oct 30 22:04:58.322114 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:58.322089    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-9f8aea/distinct-w5-9f8aea-kserve-5b564579cd-kmxzz"
Oct 30 22:04:58.322550 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:58.322520    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-0f42cb/distinct-w3-0f42cb-kserve-fb54fdd59-n6w4l"
Oct 30 22:04:58.343846 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:58.343798    4473 generic.go:356] "Generic (PLEG): container finished" podID="c9455dd0-779f-4bcb-aa4f-adb7bd780159" containerID="54e8913e351e83423e8649898d06ddc6e926c776babe72c9ad1ef1e9ef213ad8" exitCode=0
Oct 30 22:04:58.344400 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:58.344365    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-jzz7s" event={"ID":"c9455dd0-779f-4bcb-aa4f-adb7bd780159","Type":"ContainerDied","Data":"54e8913e351e83423e8649898d06ddc6e926c776babe72c9ad1ef1e9ef213ad8"}
Oct 30 22:04:58.368405 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:58.368369    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-dde1e8/distinct-2-w0-dde1e8-kserve-744857d57-ftt8b"
Oct 30 22:04:58.403950 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:58.398397    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/78f06b05-32bf-46bc-bd95-2b3a8da72b5b-credential-socket\") pod \"e2e-worker-w3-0f42cb-openshift-default-584f4c4865-l8zrp\" (UID: \"78f06b05-32bf-46bc-bd95-2b3a8da72b5b\") " pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-l8zrp"
Oct 30 22:04:58.403950 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:58.398487    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/78f06b05-32bf-46bc-bd95-2b3a8da72b5b-istio-data\") pod \"e2e-worker-w3-0f42cb-openshift-default-584f4c4865-l8zrp\" (UID: \"78f06b05-32bf-46bc-bd95-2b3a8da72b5b\") " pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-l8zrp"
Oct 30 22:04:58.403950 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:58.398634    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/78f06b05-32bf-46bc-bd95-2b3a8da72b5b-istio-envoy\") pod \"e2e-worker-w3-0f42cb-openshift-default-584f4c4865-l8zrp\" (UID: \"78f06b05-32bf-46bc-bd95-2b3a8da72b5b\") " pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-l8zrp"
Oct 30 22:04:58.403950 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:58.398762    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/78f06b05-32bf-46bc-bd95-2b3a8da72b5b-istiod-ca-cert\") pod \"e2e-worker-w3-0f42cb-openshift-default-584f4c4865-l8zrp\" (UID: \"78f06b05-32bf-46bc-bd95-2b3a8da72b5b\") " pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-l8zrp"
Oct 30 22:04:58.403950 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:58.398833    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/78f06b05-32bf-46bc-bd95-2b3a8da72b5b-istio-podinfo\") pod \"e2e-worker-w3-0f42cb-openshift-default-584f4c4865-l8zrp\" (UID: \"78f06b05-32bf-46bc-bd95-2b3a8da72b5b\") " pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-l8zrp"
Oct 30 22:04:58.403950 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:58.398878    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/78f06b05-32bf-46bc-bd95-2b3a8da72b5b-workload-certs\") pod \"e2e-worker-w3-0f42cb-openshift-default-584f4c4865-l8zrp\" (UID: \"78f06b05-32bf-46bc-bd95-2b3a8da72b5b\") " pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-l8zrp"
Oct 30 22:04:58.403950 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:58.398911    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/78f06b05-32bf-46bc-bd95-2b3a8da72b5b-workload-socket\") pod \"e2e-worker-w3-0f42cb-openshift-default-584f4c4865-l8zrp\" (UID: \"78f06b05-32bf-46bc-bd95-2b3a8da72b5b\") " pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-l8zrp"
Oct 30 22:04:58.403950 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:58.398961    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/78f06b05-32bf-46bc-bd95-2b3a8da72b5b-istio-token\") pod \"e2e-worker-w3-0f42cb-openshift-default-584f4c4865-l8zrp\" (UID: \"78f06b05-32bf-46bc-bd95-2b3a8da72b5b\") " pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-l8zrp"
Oct 30 22:04:58.403950 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:58.399024    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jcfzn\" (UniqueName: \"kubernetes.io/projected/78f06b05-32bf-46bc-bd95-2b3a8da72b5b-kube-api-access-jcfzn\") pod \"e2e-worker-w3-0f42cb-openshift-default-584f4c4865-l8zrp\" (UID: \"78f06b05-32bf-46bc-bd95-2b3a8da72b5b\") " pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-l8zrp"
Oct 30 22:04:58.422486 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:58.405436    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/78f06b05-32bf-46bc-bd95-2b3a8da72b5b-istiod-ca-cert\") pod \"e2e-worker-w3-0f42cb-openshift-default-584f4c4865-l8zrp\" (UID: \"78f06b05-32bf-46bc-bd95-2b3a8da72b5b\") " pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-l8zrp"
Oct 30 22:04:58.447869 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:58.447841    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-29cbad/premium-w6-29cbad-kserve-6c9f8d6899-qtv6c"
Oct 30 22:04:58.470594 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:58.469313    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/78f06b05-32bf-46bc-bd95-2b3a8da72b5b-credential-socket\") pod \"e2e-worker-w3-0f42cb-openshift-default-584f4c4865-l8zrp\" (UID: \"78f06b05-32bf-46bc-bd95-2b3a8da72b5b\") " pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-l8zrp"
Oct 30 22:04:58.491312 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:58.491272    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/78f06b05-32bf-46bc-bd95-2b3a8da72b5b-workload-socket\") pod \"e2e-worker-w3-0f42cb-openshift-default-584f4c4865-l8zrp\" (UID: \"78f06b05-32bf-46bc-bd95-2b3a8da72b5b\") " pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-l8zrp"
Oct 30 22:04:58.491759 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:58.491737    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/78f06b05-32bf-46bc-bd95-2b3a8da72b5b-istio-data\") pod \"e2e-worker-w3-0f42cb-openshift-default-584f4c4865-l8zrp\" (UID: \"78f06b05-32bf-46bc-bd95-2b3a8da72b5b\") " pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-l8zrp"
Oct 30 22:04:58.492129 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:58.492083    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/78f06b05-32bf-46bc-bd95-2b3a8da72b5b-workload-certs\") pod \"e2e-worker-w3-0f42cb-openshift-default-584f4c4865-l8zrp\" (UID: \"78f06b05-32bf-46bc-bd95-2b3a8da72b5b\") " pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-l8zrp"
Oct 30 22:04:58.508026 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:58.507905    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/78f06b05-32bf-46bc-bd95-2b3a8da72b5b-istio-envoy\") pod \"e2e-worker-w3-0f42cb-openshift-default-584f4c4865-l8zrp\" (UID: \"78f06b05-32bf-46bc-bd95-2b3a8da72b5b\") " pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-l8zrp"
Oct 30 22:04:58.527593 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:58.527489    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/78f06b05-32bf-46bc-bd95-2b3a8da72b5b-istio-podinfo\") pod \"e2e-worker-w3-0f42cb-openshift-default-584f4c4865-l8zrp\" (UID: \"78f06b05-32bf-46bc-bd95-2b3a8da72b5b\") " pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-l8zrp"
Oct 30 22:04:58.580041 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:58.579963    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jcfzn\" (UniqueName: \"kubernetes.io/projected/78f06b05-32bf-46bc-bd95-2b3a8da72b5b-kube-api-access-jcfzn\") pod \"e2e-worker-w3-0f42cb-openshift-default-584f4c4865-l8zrp\" (UID: \"78f06b05-32bf-46bc-bd95-2b3a8da72b5b\") " pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-l8zrp"
Oct 30 22:04:58.593209 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:58.591494    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/78f06b05-32bf-46bc-bd95-2b3a8da72b5b-istio-token\") pod \"e2e-worker-w3-0f42cb-openshift-default-584f4c4865-l8zrp\" (UID: \"78f06b05-32bf-46bc-bd95-2b3a8da72b5b\") " pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-l8zrp"
Oct 30 22:04:58.639881 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:58.639262    4473 patch_prober.go:29] interesting pod/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-8cv5d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.87:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 22:04:58.639881 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:58.639337    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-8cv5d" podUID="85b6bb64-5545-4a61-bb0d-a4851348fe30" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.87:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:04:58.660527 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:58.651549    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-l8zrp"
Oct 30 22:04:58.688993 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:58.687610    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-jzz7s"
Oct 30 22:04:58.788514 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:58.788414    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c9455dd0-779f-4bcb-aa4f-adb7bd780159-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c9455dd0-779f-4bcb-aa4f-adb7bd780159-istio-envoy\") pod \"c9455dd0-779f-4bcb-aa4f-adb7bd780159\" (UID: \"c9455dd0-779f-4bcb-aa4f-adb7bd780159\") "
Oct 30 22:04:58.789267 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:58.789230    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c9455dd0-779f-4bcb-aa4f-adb7bd780159-kube-api-access-r5pz8\" (UniqueName: \"kubernetes.io/projected/c9455dd0-779f-4bcb-aa4f-adb7bd780159-kube-api-access-r5pz8\") pod \"c9455dd0-779f-4bcb-aa4f-adb7bd780159\" (UID: \"c9455dd0-779f-4bcb-aa4f-adb7bd780159\") "
Oct 30 22:04:58.789887 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:58.789867    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c9455dd0-779f-4bcb-aa4f-adb7bd780159-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c9455dd0-779f-4bcb-aa4f-adb7bd780159-workload-certs\") pod \"c9455dd0-779f-4bcb-aa4f-adb7bd780159\" (UID: \"c9455dd0-779f-4bcb-aa4f-adb7bd780159\") "
Oct 30 22:04:58.790599 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:58.790581    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c9455dd0-779f-4bcb-aa4f-adb7bd780159-istio-token\" (UniqueName: \"kubernetes.io/projected/c9455dd0-779f-4bcb-aa4f-adb7bd780159-istio-token\") pod \"c9455dd0-779f-4bcb-aa4f-adb7bd780159\" (UID: \"c9455dd0-779f-4bcb-aa4f-adb7bd780159\") "
Oct 30 22:04:58.791144 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:58.791126    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/c9455dd0-779f-4bcb-aa4f-adb7bd780159-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c9455dd0-779f-4bcb-aa4f-adb7bd780159-istio-podinfo\") pod \"c9455dd0-779f-4bcb-aa4f-adb7bd780159\" (UID: \"c9455dd0-779f-4bcb-aa4f-adb7bd780159\") "
Oct 30 22:04:58.791607 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:58.791589    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c9455dd0-779f-4bcb-aa4f-adb7bd780159-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c9455dd0-779f-4bcb-aa4f-adb7bd780159-credential-socket\") pod \"c9455dd0-779f-4bcb-aa4f-adb7bd780159\" (UID: \"c9455dd0-779f-4bcb-aa4f-adb7bd780159\") "
Oct 30 22:04:58.791873 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:58.791856    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c9455dd0-779f-4bcb-aa4f-adb7bd780159-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c9455dd0-779f-4bcb-aa4f-adb7bd780159-workload-socket\") pod \"c9455dd0-779f-4bcb-aa4f-adb7bd780159\" (UID: \"c9455dd0-779f-4bcb-aa4f-adb7bd780159\") "
Oct 30 22:04:58.792076 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:58.792058    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c9455dd0-779f-4bcb-aa4f-adb7bd780159-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c9455dd0-779f-4bcb-aa4f-adb7bd780159-istio-data\") pod \"c9455dd0-779f-4bcb-aa4f-adb7bd780159\" (UID: \"c9455dd0-779f-4bcb-aa4f-adb7bd780159\") "
Oct 30 22:04:58.792215 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:58.792199    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c9455dd0-779f-4bcb-aa4f-adb7bd780159-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c9455dd0-779f-4bcb-aa4f-adb7bd780159-istiod-ca-cert\") pod \"c9455dd0-779f-4bcb-aa4f-adb7bd780159\" (UID: \"c9455dd0-779f-4bcb-aa4f-adb7bd780159\") "
Oct 30 22:04:58.804359 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:58.790505    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c9455dd0-779f-4bcb-aa4f-adb7bd780159-workload-certs" pod "c9455dd0-779f-4bcb-aa4f-adb7bd780159" (UID: "c9455dd0-779f-4bcb-aa4f-adb7bd780159"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:04:58.804359 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:58.798111    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c9455dd0-779f-4bcb-aa4f-adb7bd780159-istiod-ca-cert" pod "c9455dd0-779f-4bcb-aa4f-adb7bd780159" (UID: "c9455dd0-779f-4bcb-aa4f-adb7bd780159"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:04:58.804359 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:58.804114    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c9455dd0-779f-4bcb-aa4f-adb7bd780159-credential-socket" pod "c9455dd0-779f-4bcb-aa4f-adb7bd780159" (UID: "c9455dd0-779f-4bcb-aa4f-adb7bd780159"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:04:58.804982 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:58.804732    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c9455dd0-779f-4bcb-aa4f-adb7bd780159-workload-socket" pod "c9455dd0-779f-4bcb-aa4f-adb7bd780159" (UID: "c9455dd0-779f-4bcb-aa4f-adb7bd780159"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:04:58.805229 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:58.805177    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c9455dd0-779f-4bcb-aa4f-adb7bd780159-istio-data" pod "c9455dd0-779f-4bcb-aa4f-adb7bd780159" (UID: "c9455dd0-779f-4bcb-aa4f-adb7bd780159"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:04:58.843953 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:58.837007    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c9455dd0-779f-4bcb-aa4f-adb7bd780159-istio-podinfo" pod "c9455dd0-779f-4bcb-aa4f-adb7bd780159" (UID: "c9455dd0-779f-4bcb-aa4f-adb7bd780159"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:04:58.843953 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:58.841015    4473 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c9455dd0-779f-4bcb-aa4f-adb7bd780159-workload-socket\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:04:58.843953 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:58.841052    4473 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c9455dd0-779f-4bcb-aa4f-adb7bd780159-istio-data\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:04:58.843953 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:58.841067    4473 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c9455dd0-779f-4bcb-aa4f-adb7bd780159-istiod-ca-cert\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:04:58.843953 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:58.841086    4473 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c9455dd0-779f-4bcb-aa4f-adb7bd780159-workload-certs\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:04:58.843953 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:58.841107    4473 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c9455dd0-779f-4bcb-aa4f-adb7bd780159-istio-podinfo\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:04:58.843953 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:58.841122    4473 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c9455dd0-779f-4bcb-aa4f-adb7bd780159-credential-socket\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:04:58.844668 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:58.844633    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c9455dd0-779f-4bcb-aa4f-adb7bd780159-kube-api-access-r5pz8" pod "c9455dd0-779f-4bcb-aa4f-adb7bd780159" (UID: "c9455dd0-779f-4bcb-aa4f-adb7bd780159"). InnerVolumeSpecName "kube-api-access-r5pz8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:04:58.941943 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:58.927989    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-dd67d"
Oct 30 22:04:58.992475 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:58.968391    4473 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r5pz8\" (UniqueName: \"kubernetes.io/projected/c9455dd0-779f-4bcb-aa4f-adb7bd780159-kube-api-access-r5pz8\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:04:59.004783 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:59.004436    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c9455dd0-779f-4bcb-aa4f-adb7bd780159-istio-token" pod "c9455dd0-779f-4bcb-aa4f-adb7bd780159" (UID: "c9455dd0-779f-4bcb-aa4f-adb7bd780159"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:04:59.005665 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:59.005597    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c9455dd0-779f-4bcb-aa4f-adb7bd780159-istio-envoy" pod "c9455dd0-779f-4bcb-aa4f-adb7bd780159" (UID: "c9455dd0-779f-4bcb-aa4f-adb7bd780159"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:04:59.070996 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:59.070158    4473 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c9455dd0-779f-4bcb-aa4f-adb7bd780159-istio-envoy\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:04:59.070996 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:59.070190    4473 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c9455dd0-779f-4bcb-aa4f-adb7bd780159-istio-token\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:04:59.251332 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:59.251293    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-l8zrp"]
Oct 30 22:04:59.259524 ip-10-0-140-18 kubenswrapper[4473]: W1030 22:04:59.259493    4473 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78f06b05_32bf_46bc_bd95_2b3a8da72b5b.slice/crio-38bc016df66538f05af9ba50bac1b27d4b3fa91fec8d8d81b27218630f17af2a WatchSource:0}: Error finding container 38bc016df66538f05af9ba50bac1b27d4b3fa91fec8d8d81b27218630f17af2a: Status 404 returned error can't find the container with id 38bc016df66538f05af9ba50bac1b27d4b3fa91fec8d8d81b27218630f17af2a
Oct 30 22:04:59.264003 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:59.263980    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-pxzmr"
Oct 30 22:04:59.264646 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:59.264610    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:04:59.264803 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:59.264700    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:04:59.264803 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:59.264747    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:04:59.374442 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:59.374405    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-l8zrp" event={"ID":"78f06b05-32bf-46bc-bd95-2b3a8da72b5b","Type":"ContainerStarted","Data":"38bc016df66538f05af9ba50bac1b27d4b3fa91fec8d8d81b27218630f17af2a"}
Oct 30 22:04:59.381990 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:59.378145    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-jzz7s"
Oct 30 22:04:59.381990 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:59.378387    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-jzz7s" event={"ID":"c9455dd0-779f-4bcb-aa4f-adb7bd780159","Type":"ContainerDied","Data":"b28164a29bb97549e77fc7705739405ed7429a345c75f47053942a133cea573d"}
Oct 30 22:04:59.381990 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:59.378448    4473 scope.go:122] "RemoveContainer" containerID="54e8913e351e83423e8649898d06ddc6e926c776babe72c9ad1ef1e9ef213ad8"
Oct 30 22:04:59.381990 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:59.380646    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-pxzmr"
Oct 30 22:04:59.381990 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:59.381825    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-dd67d"
Oct 30 22:04:59.681526 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:59.681053    4473 patch_prober.go:29] interesting pod/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-8cv5d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.87:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 22:04:59.681526 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:59.681117    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-8cv5d" podUID="85b6bb64-5545-4a61-bb0d-a4851348fe30" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.87:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:04:59.724293 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:59.696107    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-jzz7s"]
Oct 30 22:04:59.728393 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:04:59.728342    4473 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-jzz7s"]
Oct 30 22:05:00.378734 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:00.378691    4473 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c9455dd0-779f-4bcb-aa4f-adb7bd780159" path="/var/lib/kubelet/pods/c9455dd0-779f-4bcb-aa4f-adb7bd780159/volumes"
Oct 30 22:05:00.511758 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:00.511717    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-l8zrp" event={"ID":"78f06b05-32bf-46bc-bd95-2b3a8da72b5b","Type":"ContainerStarted","Data":"e0790aefea43f190a602b7d901a99e2d06b68a40b7c1989cc0978c53b99a57d1"}
Oct 30 22:05:00.524317 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:00.524286    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-8cv5d"
Oct 30 22:05:00.580460 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:00.580431    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-8cv5d"
Oct 30 22:05:00.670889 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:00.670795    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-l8zrp"
Oct 30 22:05:00.671261 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:00.671239    4473 patch_prober.go:29] interesting pod/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-l8zrp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.90:15021/healthz/ready\": dial tcp 10.133.0.90:15021: connect: connection refused" start-of-body=
Oct 30 22:05:00.671401 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:00.671377    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-l8zrp" podUID="78f06b05-32bf-46bc-bd95-2b3a8da72b5b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.90:15021/healthz/ready\": dial tcp 10.133.0.90:15021: connect: connection refused"
Oct 30 22:05:00.828356 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:00.823593    4473 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-l8zrp" podStartSLOduration=4.823572226 podStartE2EDuration="4.823572226s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:04:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:05:00.653232891 +0000 UTC m=+1346.426601391" watchObservedRunningTime="2026-10-30 22:05:00.823572226 +0000 UTC m=+1346.596940713"
Oct 30 22:05:01.672620 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:01.672586    4473 patch_prober.go:29] interesting pod/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-l8zrp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.90:15021/healthz/ready\": dial tcp 10.133.0.90:15021: connect: connection refused" start-of-body=
Oct 30 22:05:01.672912 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:01.672878    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-l8zrp" podUID="78f06b05-32bf-46bc-bd95-2b3a8da72b5b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.90:15021/healthz/ready\": dial tcp 10.133.0.90:15021: connect: connection refused"
Oct 30 22:05:02.669702 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:02.669659    4473 patch_prober.go:29] interesting pod/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-l8zrp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.90:15021/healthz/ready\": dial tcp 10.133.0.90:15021: connect: connection refused" start-of-body=
Oct 30 22:05:02.670177 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:02.669720    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-l8zrp" podUID="78f06b05-32bf-46bc-bd95-2b3a8da72b5b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.90:15021/healthz/ready\": dial tcp 10.133.0.90:15021: connect: connection refused"
Oct 30 22:05:03.669683 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:03.669646    4473 patch_prober.go:29] interesting pod/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-l8zrp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.90:15021/healthz/ready\": dial tcp 10.133.0.90:15021: connect: connection refused" start-of-body=
Oct 30 22:05:03.669887 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:03.669710    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-l8zrp" podUID="78f06b05-32bf-46bc-bd95-2b3a8da72b5b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.90:15021/healthz/ready\": dial tcp 10.133.0.90:15021: connect: connection refused"
Oct 30 22:05:04.669039 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:04.668998    4473 patch_prober.go:29] interesting pod/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-l8zrp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.90:15021/healthz/ready\": dial tcp 10.133.0.90:15021: connect: connection refused" start-of-body=
Oct 30 22:05:04.669232 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:04.669055    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-l8zrp" podUID="78f06b05-32bf-46bc-bd95-2b3a8da72b5b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.90:15021/healthz/ready\": dial tcp 10.133.0.90:15021: connect: connection refused"
Oct 30 22:05:05.669630 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:05.669591    4473 patch_prober.go:29] interesting pod/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-l8zrp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.90:15021/healthz/ready\": dial tcp 10.133.0.90:15021: connect: connection refused" start-of-body=
Oct 30 22:05:05.670092 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:05.669665    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-l8zrp" podUID="78f06b05-32bf-46bc-bd95-2b3a8da72b5b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.90:15021/healthz/ready\": dial tcp 10.133.0.90:15021: connect: connection refused"
Oct 30 22:05:06.669481 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:06.669434    4473 patch_prober.go:29] interesting pod/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-l8zrp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.90:15021/healthz/ready\": dial tcp 10.133.0.90:15021: connect: connection refused" start-of-body=
Oct 30 22:05:06.669674 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:06.669507    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-l8zrp" podUID="78f06b05-32bf-46bc-bd95-2b3a8da72b5b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.90:15021/healthz/ready\": dial tcp 10.133.0.90:15021: connect: connection refused"
Oct 30 22:05:07.674187 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:07.674151    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-l8zrp"
Oct 30 22:05:07.674650 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:07.674609    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-l8zrp"
Oct 30 22:05:07.675375 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:07.675352    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-l8zrp"
Oct 30 22:05:10.393945 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:10.391552    4473 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-1d71f464-openshift-default-55f9d6d6bc-c4fqx"]
Oct 30 22:05:10.393945 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:10.392605    4473 cpu_manager.go:414] "containerMap: removing container" podUID="c9455dd0-779f-4bcb-aa4f-adb7bd780159" containerName="istio-proxy"
Oct 30 22:05:10.393945 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:10.392628    4473 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c9455dd0-779f-4bcb-aa4f-adb7bd780159" containerName="istio-proxy"
Oct 30 22:05:10.393945 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:10.392879    4473 memory_manager.go:365] "RemoveStaleState removing state" podUID="c9455dd0-779f-4bcb-aa4f-adb7bd780159" containerName="istio-proxy"
Oct 30 22:05:10.431946 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:10.427307    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-1d71f464-openshift-default-55f9d6d6bc-c4fqx"
Oct 30 22:05:10.501246 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:10.501206    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-1d71f464-openshift-default-55f9d6d6bc-c4fqx"]
Oct 30 22:05:10.502125 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:10.502087    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/071132ee-06b1-4b98-9905-6afd31e4289b-workload-socket\") pod \"e2e-ait-1d71f464-openshift-default-55f9d6d6bc-c4fqx\" (UID: \"071132ee-06b1-4b98-9905-6afd31e4289b\") " pod="openshift-ingress/e2e-ait-1d71f464-openshift-default-55f9d6d6bc-c4fqx"
Oct 30 22:05:10.502285 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:10.502150    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/071132ee-06b1-4b98-9905-6afd31e4289b-credential-socket\") pod \"e2e-ait-1d71f464-openshift-default-55f9d6d6bc-c4fqx\" (UID: \"071132ee-06b1-4b98-9905-6afd31e4289b\") " pod="openshift-ingress/e2e-ait-1d71f464-openshift-default-55f9d6d6bc-c4fqx"
Oct 30 22:05:10.502285 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:10.502243    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/071132ee-06b1-4b98-9905-6afd31e4289b-istio-envoy\") pod \"e2e-ait-1d71f464-openshift-default-55f9d6d6bc-c4fqx\" (UID: \"071132ee-06b1-4b98-9905-6afd31e4289b\") " pod="openshift-ingress/e2e-ait-1d71f464-openshift-default-55f9d6d6bc-c4fqx"
Oct 30 22:05:10.502285 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:10.502280    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/071132ee-06b1-4b98-9905-6afd31e4289b-istio-token\") pod \"e2e-ait-1d71f464-openshift-default-55f9d6d6bc-c4fqx\" (UID: \"071132ee-06b1-4b98-9905-6afd31e4289b\") " pod="openshift-ingress/e2e-ait-1d71f464-openshift-default-55f9d6d6bc-c4fqx"
Oct 30 22:05:10.502462 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:10.502346    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/071132ee-06b1-4b98-9905-6afd31e4289b-istio-podinfo\") pod \"e2e-ait-1d71f464-openshift-default-55f9d6d6bc-c4fqx\" (UID: \"071132ee-06b1-4b98-9905-6afd31e4289b\") " pod="openshift-ingress/e2e-ait-1d71f464-openshift-default-55f9d6d6bc-c4fqx"
Oct 30 22:05:10.502462 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:10.502415    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cf7r4\" (UniqueName: \"kubernetes.io/projected/071132ee-06b1-4b98-9905-6afd31e4289b-kube-api-access-cf7r4\") pod \"e2e-ait-1d71f464-openshift-default-55f9d6d6bc-c4fqx\" (UID: \"071132ee-06b1-4b98-9905-6afd31e4289b\") " pod="openshift-ingress/e2e-ait-1d71f464-openshift-default-55f9d6d6bc-c4fqx"
Oct 30 22:05:10.502462 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:10.502447    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/071132ee-06b1-4b98-9905-6afd31e4289b-workload-certs\") pod \"e2e-ait-1d71f464-openshift-default-55f9d6d6bc-c4fqx\" (UID: \"071132ee-06b1-4b98-9905-6afd31e4289b\") " pod="openshift-ingress/e2e-ait-1d71f464-openshift-default-55f9d6d6bc-c4fqx"
Oct 30 22:05:10.502632 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:10.502543    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/071132ee-06b1-4b98-9905-6afd31e4289b-istio-data\") pod \"e2e-ait-1d71f464-openshift-default-55f9d6d6bc-c4fqx\" (UID: \"071132ee-06b1-4b98-9905-6afd31e4289b\") " pod="openshift-ingress/e2e-ait-1d71f464-openshift-default-55f9d6d6bc-c4fqx"
Oct 30 22:05:10.502632 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:10.502610    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/071132ee-06b1-4b98-9905-6afd31e4289b-istiod-ca-cert\") pod \"e2e-ait-1d71f464-openshift-default-55f9d6d6bc-c4fqx\" (UID: \"071132ee-06b1-4b98-9905-6afd31e4289b\") " pod="openshift-ingress/e2e-ait-1d71f464-openshift-default-55f9d6d6bc-c4fqx"
Oct 30 22:05:10.544776 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:10.544628    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-1d71f464-openshift-default-dockercfg-tpmlc\""
Oct 30 22:05:10.604043 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:10.604002    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/071132ee-06b1-4b98-9905-6afd31e4289b-workload-socket\") pod \"e2e-ait-1d71f464-openshift-default-55f9d6d6bc-c4fqx\" (UID: \"071132ee-06b1-4b98-9905-6afd31e4289b\") " pod="openshift-ingress/e2e-ait-1d71f464-openshift-default-55f9d6d6bc-c4fqx"
Oct 30 22:05:10.604224 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:10.604072    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/071132ee-06b1-4b98-9905-6afd31e4289b-credential-socket\") pod \"e2e-ait-1d71f464-openshift-default-55f9d6d6bc-c4fqx\" (UID: \"071132ee-06b1-4b98-9905-6afd31e4289b\") " pod="openshift-ingress/e2e-ait-1d71f464-openshift-default-55f9d6d6bc-c4fqx"
Oct 30 22:05:10.604224 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:10.604167    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/071132ee-06b1-4b98-9905-6afd31e4289b-istio-envoy\") pod \"e2e-ait-1d71f464-openshift-default-55f9d6d6bc-c4fqx\" (UID: \"071132ee-06b1-4b98-9905-6afd31e4289b\") " pod="openshift-ingress/e2e-ait-1d71f464-openshift-default-55f9d6d6bc-c4fqx"
Oct 30 22:05:10.604224 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:10.604204    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/071132ee-06b1-4b98-9905-6afd31e4289b-istio-token\") pod \"e2e-ait-1d71f464-openshift-default-55f9d6d6bc-c4fqx\" (UID: \"071132ee-06b1-4b98-9905-6afd31e4289b\") " pod="openshift-ingress/e2e-ait-1d71f464-openshift-default-55f9d6d6bc-c4fqx"
Oct 30 22:05:10.604412 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:10.604270    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/071132ee-06b1-4b98-9905-6afd31e4289b-istio-podinfo\") pod \"e2e-ait-1d71f464-openshift-default-55f9d6d6bc-c4fqx\" (UID: \"071132ee-06b1-4b98-9905-6afd31e4289b\") " pod="openshift-ingress/e2e-ait-1d71f464-openshift-default-55f9d6d6bc-c4fqx"
Oct 30 22:05:10.604412 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:10.604348    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cf7r4\" (UniqueName: \"kubernetes.io/projected/071132ee-06b1-4b98-9905-6afd31e4289b-kube-api-access-cf7r4\") pod \"e2e-ait-1d71f464-openshift-default-55f9d6d6bc-c4fqx\" (UID: \"071132ee-06b1-4b98-9905-6afd31e4289b\") " pod="openshift-ingress/e2e-ait-1d71f464-openshift-default-55f9d6d6bc-c4fqx"
Oct 30 22:05:10.604412 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:10.604385    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/071132ee-06b1-4b98-9905-6afd31e4289b-workload-certs\") pod \"e2e-ait-1d71f464-openshift-default-55f9d6d6bc-c4fqx\" (UID: \"071132ee-06b1-4b98-9905-6afd31e4289b\") " pod="openshift-ingress/e2e-ait-1d71f464-openshift-default-55f9d6d6bc-c4fqx"
Oct 30 22:05:10.604557 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:10.604490    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/071132ee-06b1-4b98-9905-6afd31e4289b-istio-data\") pod \"e2e-ait-1d71f464-openshift-default-55f9d6d6bc-c4fqx\" (UID: \"071132ee-06b1-4b98-9905-6afd31e4289b\") " pod="openshift-ingress/e2e-ait-1d71f464-openshift-default-55f9d6d6bc-c4fqx"
Oct 30 22:05:10.604557 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:10.604526    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/071132ee-06b1-4b98-9905-6afd31e4289b-workload-socket\") pod \"e2e-ait-1d71f464-openshift-default-55f9d6d6bc-c4fqx\" (UID: \"071132ee-06b1-4b98-9905-6afd31e4289b\") " pod="openshift-ingress/e2e-ait-1d71f464-openshift-default-55f9d6d6bc-c4fqx"
Oct 30 22:05:10.604691 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:10.604567    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/071132ee-06b1-4b98-9905-6afd31e4289b-istiod-ca-cert\") pod \"e2e-ait-1d71f464-openshift-default-55f9d6d6bc-c4fqx\" (UID: \"071132ee-06b1-4b98-9905-6afd31e4289b\") " pod="openshift-ingress/e2e-ait-1d71f464-openshift-default-55f9d6d6bc-c4fqx"
Oct 30 22:05:10.604691 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:10.604569    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/071132ee-06b1-4b98-9905-6afd31e4289b-credential-socket\") pod \"e2e-ait-1d71f464-openshift-default-55f9d6d6bc-c4fqx\" (UID: \"071132ee-06b1-4b98-9905-6afd31e4289b\") " pod="openshift-ingress/e2e-ait-1d71f464-openshift-default-55f9d6d6bc-c4fqx"
Oct 30 22:05:10.605289 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:10.605238    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/071132ee-06b1-4b98-9905-6afd31e4289b-workload-certs\") pod \"e2e-ait-1d71f464-openshift-default-55f9d6d6bc-c4fqx\" (UID: \"071132ee-06b1-4b98-9905-6afd31e4289b\") " pod="openshift-ingress/e2e-ait-1d71f464-openshift-default-55f9d6d6bc-c4fqx"
Oct 30 22:05:10.605289 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:10.605266    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/071132ee-06b1-4b98-9905-6afd31e4289b-istiod-ca-cert\") pod \"e2e-ait-1d71f464-openshift-default-55f9d6d6bc-c4fqx\" (UID: \"071132ee-06b1-4b98-9905-6afd31e4289b\") " pod="openshift-ingress/e2e-ait-1d71f464-openshift-default-55f9d6d6bc-c4fqx"
Oct 30 22:05:10.605492 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:10.605305    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/071132ee-06b1-4b98-9905-6afd31e4289b-istio-data\") pod \"e2e-ait-1d71f464-openshift-default-55f9d6d6bc-c4fqx\" (UID: \"071132ee-06b1-4b98-9905-6afd31e4289b\") " pod="openshift-ingress/e2e-ait-1d71f464-openshift-default-55f9d6d6bc-c4fqx"
Oct 30 22:05:10.607308 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:10.607277    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/071132ee-06b1-4b98-9905-6afd31e4289b-istio-envoy\") pod \"e2e-ait-1d71f464-openshift-default-55f9d6d6bc-c4fqx\" (UID: \"071132ee-06b1-4b98-9905-6afd31e4289b\") " pod="openshift-ingress/e2e-ait-1d71f464-openshift-default-55f9d6d6bc-c4fqx"
Oct 30 22:05:10.610508 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:10.610479    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/071132ee-06b1-4b98-9905-6afd31e4289b-istio-podinfo\") pod \"e2e-ait-1d71f464-openshift-default-55f9d6d6bc-c4fqx\" (UID: \"071132ee-06b1-4b98-9905-6afd31e4289b\") " pod="openshift-ingress/e2e-ait-1d71f464-openshift-default-55f9d6d6bc-c4fqx"
Oct 30 22:05:10.748693 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:10.748605    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/071132ee-06b1-4b98-9905-6afd31e4289b-istio-token\") pod \"e2e-ait-1d71f464-openshift-default-55f9d6d6bc-c4fqx\" (UID: \"071132ee-06b1-4b98-9905-6afd31e4289b\") " pod="openshift-ingress/e2e-ait-1d71f464-openshift-default-55f9d6d6bc-c4fqx"
Oct 30 22:05:10.763392 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:10.763270    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cf7r4\" (UniqueName: \"kubernetes.io/projected/071132ee-06b1-4b98-9905-6afd31e4289b-kube-api-access-cf7r4\") pod \"e2e-ait-1d71f464-openshift-default-55f9d6d6bc-c4fqx\" (UID: \"071132ee-06b1-4b98-9905-6afd31e4289b\") " pod="openshift-ingress/e2e-ait-1d71f464-openshift-default-55f9d6d6bc-c4fqx"
Oct 30 22:05:11.062702 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:11.062610    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-1d71f464-openshift-default-55f9d6d6bc-c4fqx"
Oct 30 22:05:11.502264 ip-10-0-140-18 kubenswrapper[4473]: W1030 22:05:11.502230    4473 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod071132ee_06b1_4b98_9905_6afd31e4289b.slice/crio-4bae24d75b35c7070ae00d721f5f0139b8f4f610f482ac3bb2748c95d4c6ddbb WatchSource:0}: Error finding container 4bae24d75b35c7070ae00d721f5f0139b8f4f610f482ac3bb2748c95d4c6ddbb: Status 404 returned error can't find the container with id 4bae24d75b35c7070ae00d721f5f0139b8f4f610f482ac3bb2748c95d4c6ddbb
Oct 30 22:05:11.503818 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:11.503737    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-1d71f464-openshift-default-55f9d6d6bc-c4fqx"]
Oct 30 22:05:11.504849 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:11.504817    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:05:11.504987 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:11.504880    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:05:11.504987 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:11.504908    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:05:11.627358 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:11.627322    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-1d71f464-openshift-default-55f9d6d6bc-c4fqx" event={"ID":"071132ee-06b1-4b98-9905-6afd31e4289b","Type":"ContainerStarted","Data":"43a88e25ce00373dfe7cf2bb371405728c2598ac4dfd8589bb08daaf117651ea"}
Oct 30 22:05:11.627529 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:11.627382    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-1d71f464-openshift-default-55f9d6d6bc-c4fqx" event={"ID":"071132ee-06b1-4b98-9905-6afd31e4289b","Type":"ContainerStarted","Data":"4bae24d75b35c7070ae00d721f5f0139b8f4f610f482ac3bb2748c95d4c6ddbb"}
Oct 30 22:05:11.750877 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:11.750762    4473 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-1d71f464-openshift-default-55f9d6d6bc-c4fqx" podStartSLOduration=1.750741359 podStartE2EDuration="1.750741359s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:05:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:05:11.750047446 +0000 UTC m=+1357.523415950" watchObservedRunningTime="2026-10-30 22:05:11.750741359 +0000 UTC m=+1357.524109846"
Oct 30 22:05:12.063410 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:12.063326    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-1d71f464-openshift-default-55f9d6d6bc-c4fqx"
Oct 30 22:05:12.069467 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:12.069440    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-1d71f464-openshift-default-55f9d6d6bc-c4fqx"
Oct 30 22:05:12.634790 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:12.634756    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-1d71f464-openshift-default-55f9d6d6bc-c4fqx"
Oct 30 22:05:12.635877 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:12.635858    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-1d71f464-openshift-default-55f9d6d6bc-c4fqx"
Oct 30 22:05:14.499167 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:14.499138    4473 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-64086213-openshift-default-7575697f5c-dszmm"]
Oct 30 22:05:14.503623 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:14.503605    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-64086213-openshift-default-7575697f5c-dszmm"
Oct 30 22:05:14.560270 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:14.560236    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-64086213-openshift-default-dockercfg-sv2bf\""
Oct 30 22:05:14.605321 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:14.593741    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8e2ea507-72cc-48c6-87d8-6f1805fbfd43-istio-token\") pod \"e2e-ait-64086213-openshift-default-7575697f5c-dszmm\" (UID: \"8e2ea507-72cc-48c6-87d8-6f1805fbfd43\") " pod="openshift-ingress/e2e-ait-64086213-openshift-default-7575697f5c-dszmm"
Oct 30 22:05:14.624181 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:14.624146    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8e2ea507-72cc-48c6-87d8-6f1805fbfd43-istio-envoy\") pod \"e2e-ait-64086213-openshift-default-7575697f5c-dszmm\" (UID: \"8e2ea507-72cc-48c6-87d8-6f1805fbfd43\") " pod="openshift-ingress/e2e-ait-64086213-openshift-default-7575697f5c-dszmm"
Oct 30 22:05:14.624611 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:14.624585    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8e2ea507-72cc-48c6-87d8-6f1805fbfd43-istio-data\") pod \"e2e-ait-64086213-openshift-default-7575697f5c-dszmm\" (UID: \"8e2ea507-72cc-48c6-87d8-6f1805fbfd43\") " pod="openshift-ingress/e2e-ait-64086213-openshift-default-7575697f5c-dszmm"
Oct 30 22:05:14.624894 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:14.624876    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8e2ea507-72cc-48c6-87d8-6f1805fbfd43-istiod-ca-cert\") pod \"e2e-ait-64086213-openshift-default-7575697f5c-dszmm\" (UID: \"8e2ea507-72cc-48c6-87d8-6f1805fbfd43\") " pod="openshift-ingress/e2e-ait-64086213-openshift-default-7575697f5c-dszmm"
Oct 30 22:05:14.625103 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:14.625085    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sgwrm\" (UniqueName: \"kubernetes.io/projected/8e2ea507-72cc-48c6-87d8-6f1805fbfd43-kube-api-access-sgwrm\") pod \"e2e-ait-64086213-openshift-default-7575697f5c-dszmm\" (UID: \"8e2ea507-72cc-48c6-87d8-6f1805fbfd43\") " pod="openshift-ingress/e2e-ait-64086213-openshift-default-7575697f5c-dszmm"
Oct 30 22:05:14.641231 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:14.641194    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8e2ea507-72cc-48c6-87d8-6f1805fbfd43-credential-socket\") pod \"e2e-ait-64086213-openshift-default-7575697f5c-dszmm\" (UID: \"8e2ea507-72cc-48c6-87d8-6f1805fbfd43\") " pod="openshift-ingress/e2e-ait-64086213-openshift-default-7575697f5c-dszmm"
Oct 30 22:05:14.641649 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:14.641629    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8e2ea507-72cc-48c6-87d8-6f1805fbfd43-istio-podinfo\") pod \"e2e-ait-64086213-openshift-default-7575697f5c-dszmm\" (UID: \"8e2ea507-72cc-48c6-87d8-6f1805fbfd43\") " pod="openshift-ingress/e2e-ait-64086213-openshift-default-7575697f5c-dszmm"
Oct 30 22:05:14.733068 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:14.731565    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8e2ea507-72cc-48c6-87d8-6f1805fbfd43-workload-socket\") pod \"e2e-ait-64086213-openshift-default-7575697f5c-dszmm\" (UID: \"8e2ea507-72cc-48c6-87d8-6f1805fbfd43\") " pod="openshift-ingress/e2e-ait-64086213-openshift-default-7575697f5c-dszmm"
Oct 30 22:05:14.733068 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:14.731615    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8e2ea507-72cc-48c6-87d8-6f1805fbfd43-workload-certs\") pod \"e2e-ait-64086213-openshift-default-7575697f5c-dszmm\" (UID: \"8e2ea507-72cc-48c6-87d8-6f1805fbfd43\") " pod="openshift-ingress/e2e-ait-64086213-openshift-default-7575697f5c-dszmm"
Oct 30 22:05:14.733068 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:14.731942    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-64086213-openshift-default-7575697f5c-dszmm"]
Oct 30 22:05:14.997525 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:14.991986    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8e2ea507-72cc-48c6-87d8-6f1805fbfd43-istio-envoy\") pod \"e2e-ait-64086213-openshift-default-7575697f5c-dszmm\" (UID: \"8e2ea507-72cc-48c6-87d8-6f1805fbfd43\") " pod="openshift-ingress/e2e-ait-64086213-openshift-default-7575697f5c-dszmm"
Oct 30 22:05:14.997525 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:14.992079    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8e2ea507-72cc-48c6-87d8-6f1805fbfd43-istio-data\") pod \"e2e-ait-64086213-openshift-default-7575697f5c-dszmm\" (UID: \"8e2ea507-72cc-48c6-87d8-6f1805fbfd43\") " pod="openshift-ingress/e2e-ait-64086213-openshift-default-7575697f5c-dszmm"
Oct 30 22:05:14.997525 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:14.992187    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8e2ea507-72cc-48c6-87d8-6f1805fbfd43-istiod-ca-cert\") pod \"e2e-ait-64086213-openshift-default-7575697f5c-dszmm\" (UID: \"8e2ea507-72cc-48c6-87d8-6f1805fbfd43\") " pod="openshift-ingress/e2e-ait-64086213-openshift-default-7575697f5c-dszmm"
Oct 30 22:05:14.997525 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:14.992236    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sgwrm\" (UniqueName: \"kubernetes.io/projected/8e2ea507-72cc-48c6-87d8-6f1805fbfd43-kube-api-access-sgwrm\") pod \"e2e-ait-64086213-openshift-default-7575697f5c-dszmm\" (UID: \"8e2ea507-72cc-48c6-87d8-6f1805fbfd43\") " pod="openshift-ingress/e2e-ait-64086213-openshift-default-7575697f5c-dszmm"
Oct 30 22:05:14.997525 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:14.992312    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8e2ea507-72cc-48c6-87d8-6f1805fbfd43-credential-socket\") pod \"e2e-ait-64086213-openshift-default-7575697f5c-dszmm\" (UID: \"8e2ea507-72cc-48c6-87d8-6f1805fbfd43\") " pod="openshift-ingress/e2e-ait-64086213-openshift-default-7575697f5c-dszmm"
Oct 30 22:05:14.997525 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:14.992472    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8e2ea507-72cc-48c6-87d8-6f1805fbfd43-istio-podinfo\") pod \"e2e-ait-64086213-openshift-default-7575697f5c-dszmm\" (UID: \"8e2ea507-72cc-48c6-87d8-6f1805fbfd43\") " pod="openshift-ingress/e2e-ait-64086213-openshift-default-7575697f5c-dszmm"
Oct 30 22:05:14.997525 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:14.992499    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8e2ea507-72cc-48c6-87d8-6f1805fbfd43-workload-socket\") pod \"e2e-ait-64086213-openshift-default-7575697f5c-dszmm\" (UID: \"8e2ea507-72cc-48c6-87d8-6f1805fbfd43\") " pod="openshift-ingress/e2e-ait-64086213-openshift-default-7575697f5c-dszmm"
Oct 30 22:05:14.997525 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:14.992537    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8e2ea507-72cc-48c6-87d8-6f1805fbfd43-workload-certs\") pod \"e2e-ait-64086213-openshift-default-7575697f5c-dszmm\" (UID: \"8e2ea507-72cc-48c6-87d8-6f1805fbfd43\") " pod="openshift-ingress/e2e-ait-64086213-openshift-default-7575697f5c-dszmm"
Oct 30 22:05:14.997525 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:14.992567    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8e2ea507-72cc-48c6-87d8-6f1805fbfd43-istio-token\") pod \"e2e-ait-64086213-openshift-default-7575697f5c-dszmm\" (UID: \"8e2ea507-72cc-48c6-87d8-6f1805fbfd43\") " pod="openshift-ingress/e2e-ait-64086213-openshift-default-7575697f5c-dszmm"
Oct 30 22:05:14.997525 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:14.994171    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8e2ea507-72cc-48c6-87d8-6f1805fbfd43-istio-data\") pod \"e2e-ait-64086213-openshift-default-7575697f5c-dszmm\" (UID: \"8e2ea507-72cc-48c6-87d8-6f1805fbfd43\") " pod="openshift-ingress/e2e-ait-64086213-openshift-default-7575697f5c-dszmm"
Oct 30 22:05:14.997525 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:14.994941    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8e2ea507-72cc-48c6-87d8-6f1805fbfd43-istiod-ca-cert\") pod \"e2e-ait-64086213-openshift-default-7575697f5c-dszmm\" (UID: \"8e2ea507-72cc-48c6-87d8-6f1805fbfd43\") " pod="openshift-ingress/e2e-ait-64086213-openshift-default-7575697f5c-dszmm"
Oct 30 22:05:14.997525 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:14.995492    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8e2ea507-72cc-48c6-87d8-6f1805fbfd43-credential-socket\") pod \"e2e-ait-64086213-openshift-default-7575697f5c-dszmm\" (UID: \"8e2ea507-72cc-48c6-87d8-6f1805fbfd43\") " pod="openshift-ingress/e2e-ait-64086213-openshift-default-7575697f5c-dszmm"
Oct 30 22:05:14.997525 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:14.995956    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8e2ea507-72cc-48c6-87d8-6f1805fbfd43-workload-socket\") pod \"e2e-ait-64086213-openshift-default-7575697f5c-dszmm\" (UID: \"8e2ea507-72cc-48c6-87d8-6f1805fbfd43\") " pod="openshift-ingress/e2e-ait-64086213-openshift-default-7575697f5c-dszmm"
Oct 30 22:05:15.013882 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:14.999384    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8e2ea507-72cc-48c6-87d8-6f1805fbfd43-istio-podinfo\") pod \"e2e-ait-64086213-openshift-default-7575697f5c-dszmm\" (UID: \"8e2ea507-72cc-48c6-87d8-6f1805fbfd43\") " pod="openshift-ingress/e2e-ait-64086213-openshift-default-7575697f5c-dszmm"
Oct 30 22:05:15.013882 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:14.999636    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8e2ea507-72cc-48c6-87d8-6f1805fbfd43-workload-certs\") pod \"e2e-ait-64086213-openshift-default-7575697f5c-dszmm\" (UID: \"8e2ea507-72cc-48c6-87d8-6f1805fbfd43\") " pod="openshift-ingress/e2e-ait-64086213-openshift-default-7575697f5c-dszmm"
Oct 30 22:05:15.102945 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:15.102890    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8e2ea507-72cc-48c6-87d8-6f1805fbfd43-istio-envoy\") pod \"e2e-ait-64086213-openshift-default-7575697f5c-dszmm\" (UID: \"8e2ea507-72cc-48c6-87d8-6f1805fbfd43\") " pod="openshift-ingress/e2e-ait-64086213-openshift-default-7575697f5c-dszmm"
Oct 30 22:05:15.121946 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:15.117679    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sgwrm\" (UniqueName: \"kubernetes.io/projected/8e2ea507-72cc-48c6-87d8-6f1805fbfd43-kube-api-access-sgwrm\") pod \"e2e-ait-64086213-openshift-default-7575697f5c-dszmm\" (UID: \"8e2ea507-72cc-48c6-87d8-6f1805fbfd43\") " pod="openshift-ingress/e2e-ait-64086213-openshift-default-7575697f5c-dszmm"
Oct 30 22:05:15.164947 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:15.156253    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8e2ea507-72cc-48c6-87d8-6f1805fbfd43-istio-token\") pod \"e2e-ait-64086213-openshift-default-7575697f5c-dszmm\" (UID: \"8e2ea507-72cc-48c6-87d8-6f1805fbfd43\") " pod="openshift-ingress/e2e-ait-64086213-openshift-default-7575697f5c-dszmm"
Oct 30 22:05:15.441234 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:15.441204    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-64086213-openshift-default-7575697f5c-dszmm"
Oct 30 22:05:15.877340 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:15.877318    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-64086213-openshift-default-7575697f5c-dszmm"]
Oct 30 22:05:15.878637 ip-10-0-140-18 kubenswrapper[4473]: W1030 22:05:15.878607    4473 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e2ea507_72cc_48c6_87d8_6f1805fbfd43.slice/crio-2ae3763fb5b0457db5b6b31dad21d9b2bc041fd18453880905f07a906f578574 WatchSource:0}: Error finding container 2ae3763fb5b0457db5b6b31dad21d9b2bc041fd18453880905f07a906f578574: Status 404 returned error can't find the container with id 2ae3763fb5b0457db5b6b31dad21d9b2bc041fd18453880905f07a906f578574
Oct 30 22:05:15.882913 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:15.882871    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:05:15.883079 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:15.882971    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:05:15.883079 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:15.883008    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:05:16.880525 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:16.880483    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-64086213-openshift-default-7575697f5c-dszmm" event={"ID":"8e2ea507-72cc-48c6-87d8-6f1805fbfd43","Type":"ContainerStarted","Data":"1f22a165891cfc9cc1f2a87cb106cd84e60b3f6cd65674c726c8eada468802bf"}
Oct 30 22:05:16.880965 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:16.880547    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-64086213-openshift-default-7575697f5c-dszmm" event={"ID":"8e2ea507-72cc-48c6-87d8-6f1805fbfd43","Type":"ContainerStarted","Data":"2ae3763fb5b0457db5b6b31dad21d9b2bc041fd18453880905f07a906f578574"}
Oct 30 22:05:17.092231 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:17.092179    4473 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-64086213-openshift-default-7575697f5c-dszmm" podStartSLOduration=3.092163001 podStartE2EDuration="3.092163001s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:05:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:05:17.061912004 +0000 UTC m=+1362.835280492" watchObservedRunningTime="2026-10-30 22:05:17.092163001 +0000 UTC m=+1362.865531481"
Oct 30 22:05:17.454951 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:17.446139    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-64086213-openshift-default-7575697f5c-dszmm"
Oct 30 22:05:17.551229 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:17.551191    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-64086213-openshift-default-7575697f5c-dszmm"
Oct 30 22:05:17.700020 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:17.699980    4473 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-1d71f464-5df797dcdc-wwkgc"]
Oct 30 22:05:17.741004 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:17.740903    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-1d71f464-5df797dcdc-wwkgc"
Oct 30 22:05:17.818198 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:17.818165    4473 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-1d71f464\""
Oct 30 22:05:17.873946 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:17.844658    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-1d71f464-dockercfg-442wb\""
Oct 30 22:05:17.908948 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:17.885195    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-1d71f464-5df797dcdc-wwkgc"]
Oct 30 22:05:18.034994 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:18.034693    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4x7sn\" (UniqueName: \"kubernetes.io/projected/6f549a4c-acb7-46b6-85bb-65a7dd17a503-kube-api-access-4x7sn\") pod \"payload-pre-processing-e2e-ait-1d71f464-5df797dcdc-wwkgc\" (UID: \"6f549a4c-acb7-46b6-85bb-65a7dd17a503\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-1d71f464-5df797dcdc-wwkgc"
Oct 30 22:05:18.034994 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:18.034904    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6f549a4c-acb7-46b6-85bb-65a7dd17a503-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-1d71f464-5df797dcdc-wwkgc\" (UID: \"6f549a4c-acb7-46b6-85bb-65a7dd17a503\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-1d71f464-5df797dcdc-wwkgc"
Oct 30 22:05:18.056980 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:18.044854    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-64086213-openshift-default-7575697f5c-dszmm"
Oct 30 22:05:18.074296 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:18.074263    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-64086213-openshift-default-7575697f5c-dszmm"
Oct 30 22:05:18.145205 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:18.145173    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4x7sn\" (UniqueName: \"kubernetes.io/projected/6f549a4c-acb7-46b6-85bb-65a7dd17a503-kube-api-access-4x7sn\") pod \"payload-pre-processing-e2e-ait-1d71f464-5df797dcdc-wwkgc\" (UID: \"6f549a4c-acb7-46b6-85bb-65a7dd17a503\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-1d71f464-5df797dcdc-wwkgc"
Oct 30 22:05:18.147427 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:18.147393    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6f549a4c-acb7-46b6-85bb-65a7dd17a503-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-1d71f464-5df797dcdc-wwkgc\" (UID: \"6f549a4c-acb7-46b6-85bb-65a7dd17a503\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-1d71f464-5df797dcdc-wwkgc"
Oct 30 22:05:18.171142 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:18.169154    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6f549a4c-acb7-46b6-85bb-65a7dd17a503-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-1d71f464-5df797dcdc-wwkgc\" (UID: \"6f549a4c-acb7-46b6-85bb-65a7dd17a503\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-1d71f464-5df797dcdc-wwkgc"
Oct 30 22:05:18.216756 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:18.216726    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4x7sn\" (UniqueName: \"kubernetes.io/projected/6f549a4c-acb7-46b6-85bb-65a7dd17a503-kube-api-access-4x7sn\") pod \"payload-pre-processing-e2e-ait-1d71f464-5df797dcdc-wwkgc\" (UID: \"6f549a4c-acb7-46b6-85bb-65a7dd17a503\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-1d71f464-5df797dcdc-wwkgc"
Oct 30 22:05:18.470561 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:18.470530    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-1d71f464-5df797dcdc-wwkgc"
Oct 30 22:05:18.852826 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:18.852794    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-1d71f464-5df797dcdc-wwkgc"]
Oct 30 22:05:18.853354 ip-10-0-140-18 kubenswrapper[4473]: W1030 22:05:18.853326    4473 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f549a4c_acb7_46b6_85bb_65a7dd17a503.slice/crio-8a241753c70e17c9933b083ddfd9bbb8e1878cff86fa45a0229bd4fdf05e16fb WatchSource:0}: Error finding container 8a241753c70e17c9933b083ddfd9bbb8e1878cff86fa45a0229bd4fdf05e16fb: Status 404 returned error can't find the container with id 8a241753c70e17c9933b083ddfd9bbb8e1878cff86fa45a0229bd4fdf05e16fb
Oct 30 22:05:19.052138 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:19.052089    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-1d71f464-5df797dcdc-wwkgc" event={"ID":"6f549a4c-acb7-46b6-85bb-65a7dd17a503","Type":"ContainerStarted","Data":"c7830359eab1970672d4dc015db0518c7ed878f0a816d5ab9a7abffad2980ad3"}
Oct 30 22:05:19.052543 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:19.052191    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-1d71f464-5df797dcdc-wwkgc"
Oct 30 22:05:19.052543 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:19.052220    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-1d71f464-5df797dcdc-wwkgc" event={"ID":"6f549a4c-acb7-46b6-85bb-65a7dd17a503","Type":"ContainerStarted","Data":"8a241753c70e17c9933b083ddfd9bbb8e1878cff86fa45a0229bd4fdf05e16fb"}
Oct 30 22:05:19.173026 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:19.172976    4473 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-1d71f464-5df797dcdc-wwkgc" podStartSLOduration=2.172961065 podStartE2EDuration="2.172961065s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:05:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:05:19.168736303 +0000 UTC m=+1364.942104793" watchObservedRunningTime="2026-10-30 22:05:19.172961065 +0000 UTC m=+1364.946329579"
Oct 30 22:05:19.600574 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:19.600492    4473 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-97895160-openshift-default-5f45687f99-pzfmz"]
Oct 30 22:05:19.605272 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:19.605248    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-97895160-openshift-default-5f45687f99-pzfmz"
Oct 30 22:05:19.684520 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:19.679262    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-97895160-openshift-default-dockercfg-whzjp\""
Oct 30 22:05:19.720945 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:19.709041    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-97895160-openshift-default-5f45687f99-pzfmz"]
Oct 30 22:05:19.743970 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:19.727170    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e85a3a56-ad99-4c7f-bfbf-c858e78f1dd0-istio-token\") pod \"e2e-ait-97895160-openshift-default-5f45687f99-pzfmz\" (UID: \"e85a3a56-ad99-4c7f-bfbf-c858e78f1dd0\") " pod="openshift-ingress/e2e-ait-97895160-openshift-default-5f45687f99-pzfmz"
Oct 30 22:05:19.743970 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:19.727293    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e85a3a56-ad99-4c7f-bfbf-c858e78f1dd0-istiod-ca-cert\") pod \"e2e-ait-97895160-openshift-default-5f45687f99-pzfmz\" (UID: \"e85a3a56-ad99-4c7f-bfbf-c858e78f1dd0\") " pod="openshift-ingress/e2e-ait-97895160-openshift-default-5f45687f99-pzfmz"
Oct 30 22:05:19.743970 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:19.727346    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e85a3a56-ad99-4c7f-bfbf-c858e78f1dd0-credential-socket\") pod \"e2e-ait-97895160-openshift-default-5f45687f99-pzfmz\" (UID: \"e85a3a56-ad99-4c7f-bfbf-c858e78f1dd0\") " pod="openshift-ingress/e2e-ait-97895160-openshift-default-5f45687f99-pzfmz"
Oct 30 22:05:19.743970 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:19.727374    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e85a3a56-ad99-4c7f-bfbf-c858e78f1dd0-istio-podinfo\") pod \"e2e-ait-97895160-openshift-default-5f45687f99-pzfmz\" (UID: \"e85a3a56-ad99-4c7f-bfbf-c858e78f1dd0\") " pod="openshift-ingress/e2e-ait-97895160-openshift-default-5f45687f99-pzfmz"
Oct 30 22:05:19.743970 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:19.727413    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e85a3a56-ad99-4c7f-bfbf-c858e78f1dd0-workload-certs\") pod \"e2e-ait-97895160-openshift-default-5f45687f99-pzfmz\" (UID: \"e85a3a56-ad99-4c7f-bfbf-c858e78f1dd0\") " pod="openshift-ingress/e2e-ait-97895160-openshift-default-5f45687f99-pzfmz"
Oct 30 22:05:19.743970 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:19.727466    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e85a3a56-ad99-4c7f-bfbf-c858e78f1dd0-istio-envoy\") pod \"e2e-ait-97895160-openshift-default-5f45687f99-pzfmz\" (UID: \"e85a3a56-ad99-4c7f-bfbf-c858e78f1dd0\") " pod="openshift-ingress/e2e-ait-97895160-openshift-default-5f45687f99-pzfmz"
Oct 30 22:05:19.743970 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:19.727522    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lzwpp\" (UniqueName: \"kubernetes.io/projected/e85a3a56-ad99-4c7f-bfbf-c858e78f1dd0-kube-api-access-lzwpp\") pod \"e2e-ait-97895160-openshift-default-5f45687f99-pzfmz\" (UID: \"e85a3a56-ad99-4c7f-bfbf-c858e78f1dd0\") " pod="openshift-ingress/e2e-ait-97895160-openshift-default-5f45687f99-pzfmz"
Oct 30 22:05:19.743970 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:19.727634    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e85a3a56-ad99-4c7f-bfbf-c858e78f1dd0-workload-socket\") pod \"e2e-ait-97895160-openshift-default-5f45687f99-pzfmz\" (UID: \"e85a3a56-ad99-4c7f-bfbf-c858e78f1dd0\") " pod="openshift-ingress/e2e-ait-97895160-openshift-default-5f45687f99-pzfmz"
Oct 30 22:05:19.743970 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:19.727819    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e85a3a56-ad99-4c7f-bfbf-c858e78f1dd0-istio-data\") pod \"e2e-ait-97895160-openshift-default-5f45687f99-pzfmz\" (UID: \"e85a3a56-ad99-4c7f-bfbf-c858e78f1dd0\") " pod="openshift-ingress/e2e-ait-97895160-openshift-default-5f45687f99-pzfmz"
Oct 30 22:05:19.919006 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:19.843633    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e85a3a56-ad99-4c7f-bfbf-c858e78f1dd0-istio-data\") pod \"e2e-ait-97895160-openshift-default-5f45687f99-pzfmz\" (UID: \"e85a3a56-ad99-4c7f-bfbf-c858e78f1dd0\") " pod="openshift-ingress/e2e-ait-97895160-openshift-default-5f45687f99-pzfmz"
Oct 30 22:05:19.919006 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:19.843712    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e85a3a56-ad99-4c7f-bfbf-c858e78f1dd0-istio-token\") pod \"e2e-ait-97895160-openshift-default-5f45687f99-pzfmz\" (UID: \"e85a3a56-ad99-4c7f-bfbf-c858e78f1dd0\") " pod="openshift-ingress/e2e-ait-97895160-openshift-default-5f45687f99-pzfmz"
Oct 30 22:05:19.919006 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:19.843814    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e85a3a56-ad99-4c7f-bfbf-c858e78f1dd0-istiod-ca-cert\") pod \"e2e-ait-97895160-openshift-default-5f45687f99-pzfmz\" (UID: \"e85a3a56-ad99-4c7f-bfbf-c858e78f1dd0\") " pod="openshift-ingress/e2e-ait-97895160-openshift-default-5f45687f99-pzfmz"
Oct 30 22:05:19.919006 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:19.843862    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e85a3a56-ad99-4c7f-bfbf-c858e78f1dd0-credential-socket\") pod \"e2e-ait-97895160-openshift-default-5f45687f99-pzfmz\" (UID: \"e85a3a56-ad99-4c7f-bfbf-c858e78f1dd0\") " pod="openshift-ingress/e2e-ait-97895160-openshift-default-5f45687f99-pzfmz"
Oct 30 22:05:19.919006 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:19.843892    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e85a3a56-ad99-4c7f-bfbf-c858e78f1dd0-istio-podinfo\") pod \"e2e-ait-97895160-openshift-default-5f45687f99-pzfmz\" (UID: \"e85a3a56-ad99-4c7f-bfbf-c858e78f1dd0\") " pod="openshift-ingress/e2e-ait-97895160-openshift-default-5f45687f99-pzfmz"
Oct 30 22:05:19.919006 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:19.843951    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e85a3a56-ad99-4c7f-bfbf-c858e78f1dd0-workload-certs\") pod \"e2e-ait-97895160-openshift-default-5f45687f99-pzfmz\" (UID: \"e85a3a56-ad99-4c7f-bfbf-c858e78f1dd0\") " pod="openshift-ingress/e2e-ait-97895160-openshift-default-5f45687f99-pzfmz"
Oct 30 22:05:19.919006 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:19.844022    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e85a3a56-ad99-4c7f-bfbf-c858e78f1dd0-istio-envoy\") pod \"e2e-ait-97895160-openshift-default-5f45687f99-pzfmz\" (UID: \"e85a3a56-ad99-4c7f-bfbf-c858e78f1dd0\") " pod="openshift-ingress/e2e-ait-97895160-openshift-default-5f45687f99-pzfmz"
Oct 30 22:05:19.919006 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:19.844076    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lzwpp\" (UniqueName: \"kubernetes.io/projected/e85a3a56-ad99-4c7f-bfbf-c858e78f1dd0-kube-api-access-lzwpp\") pod \"e2e-ait-97895160-openshift-default-5f45687f99-pzfmz\" (UID: \"e85a3a56-ad99-4c7f-bfbf-c858e78f1dd0\") " pod="openshift-ingress/e2e-ait-97895160-openshift-default-5f45687f99-pzfmz"
Oct 30 22:05:19.919006 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:19.844182    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e85a3a56-ad99-4c7f-bfbf-c858e78f1dd0-workload-socket\") pod \"e2e-ait-97895160-openshift-default-5f45687f99-pzfmz\" (UID: \"e85a3a56-ad99-4c7f-bfbf-c858e78f1dd0\") " pod="openshift-ingress/e2e-ait-97895160-openshift-default-5f45687f99-pzfmz"
Oct 30 22:05:19.919006 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:19.844660    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e85a3a56-ad99-4c7f-bfbf-c858e78f1dd0-workload-socket\") pod \"e2e-ait-97895160-openshift-default-5f45687f99-pzfmz\" (UID: \"e85a3a56-ad99-4c7f-bfbf-c858e78f1dd0\") " pod="openshift-ingress/e2e-ait-97895160-openshift-default-5f45687f99-pzfmz"
Oct 30 22:05:19.924356 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:19.920983    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e85a3a56-ad99-4c7f-bfbf-c858e78f1dd0-credential-socket\") pod \"e2e-ait-97895160-openshift-default-5f45687f99-pzfmz\" (UID: \"e85a3a56-ad99-4c7f-bfbf-c858e78f1dd0\") " pod="openshift-ingress/e2e-ait-97895160-openshift-default-5f45687f99-pzfmz"
Oct 30 22:05:19.924356 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:19.921274    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e85a3a56-ad99-4c7f-bfbf-c858e78f1dd0-istio-data\") pod \"e2e-ait-97895160-openshift-default-5f45687f99-pzfmz\" (UID: \"e85a3a56-ad99-4c7f-bfbf-c858e78f1dd0\") " pod="openshift-ingress/e2e-ait-97895160-openshift-default-5f45687f99-pzfmz"
Oct 30 22:05:19.924356 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:19.921353    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e85a3a56-ad99-4c7f-bfbf-c858e78f1dd0-workload-certs\") pod \"e2e-ait-97895160-openshift-default-5f45687f99-pzfmz\" (UID: \"e85a3a56-ad99-4c7f-bfbf-c858e78f1dd0\") " pod="openshift-ingress/e2e-ait-97895160-openshift-default-5f45687f99-pzfmz"
Oct 30 22:05:19.924356 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:19.922306    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e85a3a56-ad99-4c7f-bfbf-c858e78f1dd0-istiod-ca-cert\") pod \"e2e-ait-97895160-openshift-default-5f45687f99-pzfmz\" (UID: \"e85a3a56-ad99-4c7f-bfbf-c858e78f1dd0\") " pod="openshift-ingress/e2e-ait-97895160-openshift-default-5f45687f99-pzfmz"
Oct 30 22:05:20.016950 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:20.015835    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lzwpp\" (UniqueName: \"kubernetes.io/projected/e85a3a56-ad99-4c7f-bfbf-c858e78f1dd0-kube-api-access-lzwpp\") pod \"e2e-ait-97895160-openshift-default-5f45687f99-pzfmz\" (UID: \"e85a3a56-ad99-4c7f-bfbf-c858e78f1dd0\") " pod="openshift-ingress/e2e-ait-97895160-openshift-default-5f45687f99-pzfmz"
Oct 30 22:05:20.016950 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:20.016684    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e85a3a56-ad99-4c7f-bfbf-c858e78f1dd0-istio-podinfo\") pod \"e2e-ait-97895160-openshift-default-5f45687f99-pzfmz\" (UID: \"e85a3a56-ad99-4c7f-bfbf-c858e78f1dd0\") " pod="openshift-ingress/e2e-ait-97895160-openshift-default-5f45687f99-pzfmz"
Oct 30 22:05:20.023235 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:20.017364    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e85a3a56-ad99-4c7f-bfbf-c858e78f1dd0-istio-envoy\") pod \"e2e-ait-97895160-openshift-default-5f45687f99-pzfmz\" (UID: \"e85a3a56-ad99-4c7f-bfbf-c858e78f1dd0\") " pod="openshift-ingress/e2e-ait-97895160-openshift-default-5f45687f99-pzfmz"
Oct 30 22:05:20.023235 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:20.021327    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e85a3a56-ad99-4c7f-bfbf-c858e78f1dd0-istio-token\") pod \"e2e-ait-97895160-openshift-default-5f45687f99-pzfmz\" (UID: \"e85a3a56-ad99-4c7f-bfbf-c858e78f1dd0\") " pod="openshift-ingress/e2e-ait-97895160-openshift-default-5f45687f99-pzfmz"
Oct 30 22:05:20.229069 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:20.223244    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-97895160-openshift-default-5f45687f99-pzfmz"
Oct 30 22:05:20.329862 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:05:20.329827    4473 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fncvg" containerName="extractor"
Oct 30 22:05:20.711699 ip-10-0-140-18 kubenswrapper[4473]: W1030 22:05:20.711670    4473 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode85a3a56_ad99_4c7f_bfbf_c858e78f1dd0.slice/crio-b6ae8e533d44776034721d79d6a9f7c4948d35d7ce0af8d588c77139e86bedd5 WatchSource:0}: Error finding container b6ae8e533d44776034721d79d6a9f7c4948d35d7ce0af8d588c77139e86bedd5: Status 404 returned error can't find the container with id b6ae8e533d44776034721d79d6a9f7c4948d35d7ce0af8d588c77139e86bedd5
Oct 30 22:05:20.712169 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:20.712141    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-97895160-openshift-default-5f45687f99-pzfmz"]
Oct 30 22:05:20.714435 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:20.714398    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:05:20.714539 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:20.714465    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:05:20.714539 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:20.714494    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:05:21.173326 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:21.173288    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-97895160-openshift-default-5f45687f99-pzfmz" event={"ID":"e85a3a56-ad99-4c7f-bfbf-c858e78f1dd0","Type":"ContainerStarted","Data":"56b99d959e478fff249b8fa8302ee589f3144a062ebd32d58e9aa8e87605f699"}
Oct 30 22:05:21.173596 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:21.173350    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-97895160-openshift-default-5f45687f99-pzfmz" event={"ID":"e85a3a56-ad99-4c7f-bfbf-c858e78f1dd0","Type":"ContainerStarted","Data":"b6ae8e533d44776034721d79d6a9f7c4948d35d7ce0af8d588c77139e86bedd5"}
Oct 30 22:05:21.224231 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:21.224200    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-97895160-openshift-default-5f45687f99-pzfmz"
Oct 30 22:05:21.230378 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:21.230351    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-97895160-openshift-default-5f45687f99-pzfmz"
Oct 30 22:05:21.298534 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:21.298483    4473 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-97895160-openshift-default-5f45687f99-pzfmz" podStartSLOduration=2.298467331 podStartE2EDuration="2.298467331s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:05:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:05:21.296422162 +0000 UTC m=+1367.069790649" watchObservedRunningTime="2026-10-30 22:05:21.298467331 +0000 UTC m=+1367.071835819"
Oct 30 22:05:22.180474 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:22.180439    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-97895160-openshift-default-5f45687f99-pzfmz"
Oct 30 22:05:22.181469 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:22.181449    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-97895160-openshift-default-5f45687f99-pzfmz"
Oct 30 22:05:24.942785 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:24.942748    4473 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-gsc42"]
Oct 30 22:05:24.947345 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:24.947304    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-gsc42"
Oct 30 22:05:25.016935 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:25.016895    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c982f07-a1d8-4b18-a065-4e53f7e5022a-workload-socket\") pod \"e2e-worker-w2-0c99b1-openshift-default-967855977-gsc42\" (UID: \"3c982f07-a1d8-4b18-a065-4e53f7e5022a\") " pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-gsc42"
Oct 30 22:05:25.017107 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:25.017019    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3c982f07-a1d8-4b18-a065-4e53f7e5022a-istio-envoy\") pod \"e2e-worker-w2-0c99b1-openshift-default-967855977-gsc42\" (UID: \"3c982f07-a1d8-4b18-a065-4e53f7e5022a\") " pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-gsc42"
Oct 30 22:05:25.017177 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:25.017162    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3c982f07-a1d8-4b18-a065-4e53f7e5022a-istio-data\") pod \"e2e-worker-w2-0c99b1-openshift-default-967855977-gsc42\" (UID: \"3c982f07-a1d8-4b18-a065-4e53f7e5022a\") " pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-gsc42"
Oct 30 22:05:25.017234 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:25.017211    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c982f07-a1d8-4b18-a065-4e53f7e5022a-credential-socket\") pod \"e2e-worker-w2-0c99b1-openshift-default-967855977-gsc42\" (UID: \"3c982f07-a1d8-4b18-a065-4e53f7e5022a\") " pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-gsc42"
Oct 30 22:05:25.017308 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:25.017285    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3c982f07-a1d8-4b18-a065-4e53f7e5022a-workload-certs\") pod \"e2e-worker-w2-0c99b1-openshift-default-967855977-gsc42\" (UID: \"3c982f07-a1d8-4b18-a065-4e53f7e5022a\") " pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-gsc42"
Oct 30 22:05:25.017388 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:25.017374    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3c982f07-a1d8-4b18-a065-4e53f7e5022a-istiod-ca-cert\") pod \"e2e-worker-w2-0c99b1-openshift-default-967855977-gsc42\" (UID: \"3c982f07-a1d8-4b18-a065-4e53f7e5022a\") " pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-gsc42"
Oct 30 22:05:25.017506 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:25.017492    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3c982f07-a1d8-4b18-a065-4e53f7e5022a-istio-podinfo\") pod \"e2e-worker-w2-0c99b1-openshift-default-967855977-gsc42\" (UID: \"3c982f07-a1d8-4b18-a065-4e53f7e5022a\") " pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-gsc42"
Oct 30 22:05:25.017609 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:25.017595    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3c982f07-a1d8-4b18-a065-4e53f7e5022a-istio-token\") pod \"e2e-worker-w2-0c99b1-openshift-default-967855977-gsc42\" (UID: \"3c982f07-a1d8-4b18-a065-4e53f7e5022a\") " pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-gsc42"
Oct 30 22:05:25.017679 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:25.017649    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2jgnc\" (UniqueName: \"kubernetes.io/projected/3c982f07-a1d8-4b18-a065-4e53f7e5022a-kube-api-access-2jgnc\") pod \"e2e-worker-w2-0c99b1-openshift-default-967855977-gsc42\" (UID: \"3c982f07-a1d8-4b18-a065-4e53f7e5022a\") " pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-gsc42"
Oct 30 22:05:25.118912 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:25.118875    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3c982f07-a1d8-4b18-a065-4e53f7e5022a-istio-data\") pod \"e2e-worker-w2-0c99b1-openshift-default-967855977-gsc42\" (UID: \"3c982f07-a1d8-4b18-a065-4e53f7e5022a\") " pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-gsc42"
Oct 30 22:05:25.119123 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:25.118972    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c982f07-a1d8-4b18-a065-4e53f7e5022a-credential-socket\") pod \"e2e-worker-w2-0c99b1-openshift-default-967855977-gsc42\" (UID: \"3c982f07-a1d8-4b18-a065-4e53f7e5022a\") " pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-gsc42"
Oct 30 22:05:25.119123 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:25.118992    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3c982f07-a1d8-4b18-a065-4e53f7e5022a-workload-certs\") pod \"e2e-worker-w2-0c99b1-openshift-default-967855977-gsc42\" (UID: \"3c982f07-a1d8-4b18-a065-4e53f7e5022a\") " pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-gsc42"
Oct 30 22:05:25.119123 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:25.119040    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3c982f07-a1d8-4b18-a065-4e53f7e5022a-istiod-ca-cert\") pod \"e2e-worker-w2-0c99b1-openshift-default-967855977-gsc42\" (UID: \"3c982f07-a1d8-4b18-a065-4e53f7e5022a\") " pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-gsc42"
Oct 30 22:05:25.119350 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:25.119139    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3c982f07-a1d8-4b18-a065-4e53f7e5022a-istio-podinfo\") pod \"e2e-worker-w2-0c99b1-openshift-default-967855977-gsc42\" (UID: \"3c982f07-a1d8-4b18-a065-4e53f7e5022a\") " pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-gsc42"
Oct 30 22:05:25.119350 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:25.119239    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3c982f07-a1d8-4b18-a065-4e53f7e5022a-istio-token\") pod \"e2e-worker-w2-0c99b1-openshift-default-967855977-gsc42\" (UID: \"3c982f07-a1d8-4b18-a065-4e53f7e5022a\") " pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-gsc42"
Oct 30 22:05:25.119350 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:25.119266    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2jgnc\" (UniqueName: \"kubernetes.io/projected/3c982f07-a1d8-4b18-a065-4e53f7e5022a-kube-api-access-2jgnc\") pod \"e2e-worker-w2-0c99b1-openshift-default-967855977-gsc42\" (UID: \"3c982f07-a1d8-4b18-a065-4e53f7e5022a\") " pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-gsc42"
Oct 30 22:05:25.119500 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:25.119357    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c982f07-a1d8-4b18-a065-4e53f7e5022a-credential-socket\") pod \"e2e-worker-w2-0c99b1-openshift-default-967855977-gsc42\" (UID: \"3c982f07-a1d8-4b18-a065-4e53f7e5022a\") " pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-gsc42"
Oct 30 22:05:25.119500 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:25.119374    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c982f07-a1d8-4b18-a065-4e53f7e5022a-workload-socket\") pod \"e2e-worker-w2-0c99b1-openshift-default-967855977-gsc42\" (UID: \"3c982f07-a1d8-4b18-a065-4e53f7e5022a\") " pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-gsc42"
Oct 30 22:05:25.119500 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:25.119461    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3c982f07-a1d8-4b18-a065-4e53f7e5022a-istio-envoy\") pod \"e2e-worker-w2-0c99b1-openshift-default-967855977-gsc42\" (UID: \"3c982f07-a1d8-4b18-a065-4e53f7e5022a\") " pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-gsc42"
Oct 30 22:05:25.119737 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:25.119713    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3c982f07-a1d8-4b18-a065-4e53f7e5022a-istio-data\") pod \"e2e-worker-w2-0c99b1-openshift-default-967855977-gsc42\" (UID: \"3c982f07-a1d8-4b18-a065-4e53f7e5022a\") " pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-gsc42"
Oct 30 22:05:25.120106 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:25.120081    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3c982f07-a1d8-4b18-a065-4e53f7e5022a-workload-certs\") pod \"e2e-worker-w2-0c99b1-openshift-default-967855977-gsc42\" (UID: \"3c982f07-a1d8-4b18-a065-4e53f7e5022a\") " pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-gsc42"
Oct 30 22:05:25.120301 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:25.120144    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c982f07-a1d8-4b18-a065-4e53f7e5022a-workload-socket\") pod \"e2e-worker-w2-0c99b1-openshift-default-967855977-gsc42\" (UID: \"3c982f07-a1d8-4b18-a065-4e53f7e5022a\") " pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-gsc42"
Oct 30 22:05:25.120451 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:25.120369    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3c982f07-a1d8-4b18-a065-4e53f7e5022a-istiod-ca-cert\") pod \"e2e-worker-w2-0c99b1-openshift-default-967855977-gsc42\" (UID: \"3c982f07-a1d8-4b18-a065-4e53f7e5022a\") " pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-gsc42"
Oct 30 22:05:25.122107 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:25.122087    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3c982f07-a1d8-4b18-a065-4e53f7e5022a-istio-podinfo\") pod \"e2e-worker-w2-0c99b1-openshift-default-967855977-gsc42\" (UID: \"3c982f07-a1d8-4b18-a065-4e53f7e5022a\") " pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-gsc42"
Oct 30 22:05:25.122197 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:25.122138    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3c982f07-a1d8-4b18-a065-4e53f7e5022a-istio-envoy\") pod \"e2e-worker-w2-0c99b1-openshift-default-967855977-gsc42\" (UID: \"3c982f07-a1d8-4b18-a065-4e53f7e5022a\") " pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-gsc42"
Oct 30 22:05:25.154110 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:25.154077    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-gsc42"]
Oct 30 22:05:25.177312 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:25.177285    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3c982f07-a1d8-4b18-a065-4e53f7e5022a-istio-token\") pod \"e2e-worker-w2-0c99b1-openshift-default-967855977-gsc42\" (UID: \"3c982f07-a1d8-4b18-a065-4e53f7e5022a\") " pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-gsc42"
Oct 30 22:05:25.177524 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:25.177504    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2jgnc\" (UniqueName: \"kubernetes.io/projected/3c982f07-a1d8-4b18-a065-4e53f7e5022a-kube-api-access-2jgnc\") pod \"e2e-worker-w2-0c99b1-openshift-default-967855977-gsc42\" (UID: \"3c982f07-a1d8-4b18-a065-4e53f7e5022a\") " pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-gsc42"
Oct 30 22:05:25.264291 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:25.264173    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-gsc42"
Oct 30 22:05:25.702393 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:25.702362    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-gsc42"]
Oct 30 22:05:25.704709 ip-10-0-140-18 kubenswrapper[4473]: W1030 22:05:25.704670    4473 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c982f07_a1d8_4b18_a065_4e53f7e5022a.slice/crio-89d95c9ebf5a9d57f060a0f595dc5eef3f15774551d43c4ac62550e7a4d6980a WatchSource:0}: Error finding container 89d95c9ebf5a9d57f060a0f595dc5eef3f15774551d43c4ac62550e7a4d6980a: Status 404 returned error can't find the container with id 89d95c9ebf5a9d57f060a0f595dc5eef3f15774551d43c4ac62550e7a4d6980a
Oct 30 22:05:25.707612 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:25.707577    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:05:25.707697 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:25.707647    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:05:25.707697 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:25.707676    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:05:26.658479 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:26.658434    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-gsc42" event={"ID":"3c982f07-a1d8-4b18-a065-4e53f7e5022a","Type":"ContainerStarted","Data":"ce6d303a42496486091e3e9f8d8f2e2895ecdcbfe30841234474479823fceed3"}
Oct 30 22:05:26.658907 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:26.658503    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-gsc42" event={"ID":"3c982f07-a1d8-4b18-a065-4e53f7e5022a","Type":"ContainerStarted","Data":"89d95c9ebf5a9d57f060a0f595dc5eef3f15774551d43c4ac62550e7a4d6980a"}
Oct 30 22:05:26.892807 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:26.892749    4473 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-gsc42" podStartSLOduration=2.892735472 podStartE2EDuration="2.892735472s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:05:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:05:26.84142058 +0000 UTC m=+1372.614789093" watchObservedRunningTime="2026-10-30 22:05:26.892735472 +0000 UTC m=+1372.666103992"
Oct 30 22:05:27.264450 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:27.264410    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-gsc42"
Oct 30 22:05:27.266443 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:27.266412    4473 patch_prober.go:29] interesting pod/e2e-worker-w2-0c99b1-openshift-default-967855977-gsc42 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.95:15021/healthz/ready\": dial tcp 10.133.0.95:15021: connect: connection refused" start-of-body=
Oct 30 22:05:27.266603 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:27.266476    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-gsc42" podUID="3c982f07-a1d8-4b18-a065-4e53f7e5022a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.95:15021/healthz/ready\": dial tcp 10.133.0.95:15021: connect: connection refused"
Oct 30 22:05:27.901963 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:27.901694    4473 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-97895160-784c6ff67f-6xks4"]
Oct 30 22:05:27.925940 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:27.922361    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-97895160-784c6ff67f-6xks4"
Oct 30 22:05:27.980268 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:27.979608    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fe4f0d92-1deb-445e-b700-2e62abd9e4e3-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-97895160-784c6ff67f-6xks4\" (UID: \"fe4f0d92-1deb-445e-b700-2e62abd9e4e3\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-97895160-784c6ff67f-6xks4"
Oct 30 22:05:27.980268 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:27.979842    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ql6hs\" (UniqueName: \"kubernetes.io/projected/fe4f0d92-1deb-445e-b700-2e62abd9e4e3-kube-api-access-ql6hs\") pod \"payload-pre-processing-e2e-ait-97895160-784c6ff67f-6xks4\" (UID: \"fe4f0d92-1deb-445e-b700-2e62abd9e4e3\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-97895160-784c6ff67f-6xks4"
Oct 30 22:05:28.001948 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:27.985501    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-97895160-784c6ff67f-6xks4"]
Oct 30 22:05:28.021328 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:28.021293    4473 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-97895160\""
Oct 30 22:05:28.072961 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:28.021742    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-97895160-dockercfg-qxxlw\""
Oct 30 22:05:28.127319 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:28.127277    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ql6hs\" (UniqueName: \"kubernetes.io/projected/fe4f0d92-1deb-445e-b700-2e62abd9e4e3-kube-api-access-ql6hs\") pod \"payload-pre-processing-e2e-ait-97895160-784c6ff67f-6xks4\" (UID: \"fe4f0d92-1deb-445e-b700-2e62abd9e4e3\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-97895160-784c6ff67f-6xks4"
Oct 30 22:05:28.127709 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:28.127683    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fe4f0d92-1deb-445e-b700-2e62abd9e4e3-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-97895160-784c6ff67f-6xks4\" (UID: \"fe4f0d92-1deb-445e-b700-2e62abd9e4e3\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-97895160-784c6ff67f-6xks4"
Oct 30 22:05:28.129600 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:28.129575    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fe4f0d92-1deb-445e-b700-2e62abd9e4e3-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-97895160-784c6ff67f-6xks4\" (UID: \"fe4f0d92-1deb-445e-b700-2e62abd9e4e3\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-97895160-784c6ff67f-6xks4"
Oct 30 22:05:28.420507 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:28.420470    4473 patch_prober.go:29] interesting pod/e2e-worker-w2-0c99b1-openshift-default-967855977-gsc42 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.95:15021/healthz/ready\": dial tcp 10.133.0.95:15021: connect: connection refused" start-of-body=
Oct 30 22:05:28.420822 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:28.420779    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-gsc42" podUID="3c982f07-a1d8-4b18-a065-4e53f7e5022a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.95:15021/healthz/ready\": dial tcp 10.133.0.95:15021: connect: connection refused"
Oct 30 22:05:28.593595 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:28.577023    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ql6hs\" (UniqueName: \"kubernetes.io/projected/fe4f0d92-1deb-445e-b700-2e62abd9e4e3-kube-api-access-ql6hs\") pod \"payload-pre-processing-e2e-ait-97895160-784c6ff67f-6xks4\" (UID: \"fe4f0d92-1deb-445e-b700-2e62abd9e4e3\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-97895160-784c6ff67f-6xks4"
Oct 30 22:05:28.869333 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:28.869272    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-97895160-784c6ff67f-6xks4"
Oct 30 22:05:29.270017 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:29.268338    4473 patch_prober.go:29] interesting pod/e2e-worker-w2-0c99b1-openshift-default-967855977-gsc42 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.95:15021/healthz/ready\": dial tcp 10.133.0.95:15021: connect: connection refused" start-of-body=
Oct 30 22:05:29.270017 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:29.268401    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-gsc42" podUID="3c982f07-a1d8-4b18-a065-4e53f7e5022a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.95:15021/healthz/ready\": dial tcp 10.133.0.95:15021: connect: connection refused"
Oct 30 22:05:29.627466 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:29.618868    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-97895160-784c6ff67f-6xks4"]
Oct 30 22:05:29.870034 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:29.862312    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-97895160-784c6ff67f-6xks4" event={"ID":"fe4f0d92-1deb-445e-b700-2e62abd9e4e3","Type":"ContainerStarted","Data":"b361fcc52cb42837e87768ade8e3bb08deec5c459380cf41505e6673ffefcc3b"}
Oct 30 22:05:30.081944 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:30.079431    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-0f42cb/distinct-w3-0f42cb-kserve-fb54fdd59-n6w4l" podUID="ff3c2b25-08aa-4b70-bf57-9cf4a11d05ed" containerName="main" probeResult="failure" output="Get \"https://10.133.0.73:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:05:30.135999 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:30.125027    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-9f8aea/distinct-w5-9f8aea-kserve-5b564579cd-kmxzz" podUID="b11e8b11-334f-4a9e-98c2-b0164567a8af" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:05:30.182048 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:30.181185    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-1d71f464-5df797dcdc-wwkgc"
Oct 30 22:05:30.213862 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:30.213808    4473 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w6-29cbad/premium-w6-29cbad-kserve-6c9f8d6899-qtv6c" podUID="381b11f6-3fde-4de7-9ad3-7c0891e76ba1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.60:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:05:30.312394 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:30.307348    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-29cbad/premium-w6-29cbad-kserve-6c9f8d6899-qtv6c" podUID="381b11f6-3fde-4de7-9ad3-7c0891e76ba1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.60:8000/ready\": context deadline exceeded"
Oct 30 22:05:30.327960 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:30.327507    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-0f42cb/distinct-2-w3-0f42cb-kserve-fc477897b-jk8qc" podUID="61b49182-057e-41af-b609-1a6abf82c197" containerName="main" probeResult="failure" output="Get \"https://10.133.0.80:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:05:30.415015 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:30.409883    4473 patch_prober.go:29] interesting pod/e2e-worker-w2-0c99b1-openshift-default-967855977-gsc42 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.95:15021/healthz/ready\": dial tcp 10.133.0.95:15021: connect: connection refused" start-of-body=
Oct 30 22:05:30.415015 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:30.409982    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-gsc42" podUID="3c982f07-a1d8-4b18-a065-4e53f7e5022a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.95:15021/healthz/ready\": dial tcp 10.133.0.95:15021: connect: connection refused"
Oct 30 22:05:30.527441 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:30.527387    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-0c99b1/distinct-2-w2-0c99b1-kserve-7466864658-8z7fk" podUID="89c09555-cf53-49a7-9c3b-95d2da2a293a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.82:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:05:30.835899 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:30.835851    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-dde1e8/distinct-2-w0-dde1e8-kserve-744857d57-ftt8b" podUID="577b020d-2a6b-4020-863c-cf19d4f0ca3b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.79:8000/ready\": context deadline exceeded"
Oct 30 22:05:31.083030 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:31.047582    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-dde1e8/distinct-w0-dde1e8-kserve-75b68b4c5d-hts2k" podUID="9d72e7c9-657e-4278-8ed9-53adb75bd530" containerName="main" probeResult="failure" output="Get \"https://10.133.0.72:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:05:31.087803 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:31.087767    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-0f42cb/distinct-w3-0f42cb-kserve-fb54fdd59-n6w4l" podUID="ff3c2b25-08aa-4b70-bf57-9cf4a11d05ed" containerName="main" probeResult="failure" output="Get \"https://10.133.0.73:8000/ready\": context deadline exceeded"
Oct 30 22:05:31.088479 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:31.088454    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-0f42cb/distinct-w3-0f42cb-kserve-fb54fdd59-n6w4l"
Oct 30 22:05:31.126058 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:31.126014    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-9f8aea/distinct-w5-9f8aea-kserve-5b564579cd-kmxzz" podUID="b11e8b11-334f-4a9e-98c2-b0164567a8af" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:05:31.126698 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:31.126672    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-9f8aea/distinct-w5-9f8aea-kserve-5b564579cd-kmxzz"
Oct 30 22:05:31.278957 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:31.278132    4473 patch_prober.go:29] interesting pod/e2e-worker-w2-0c99b1-openshift-default-967855977-gsc42 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.95:15021/healthz/ready\": dial tcp 10.133.0.95:15021: connect: connection refused" start-of-body=
Oct 30 22:05:31.278957 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:31.278196    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-gsc42" podUID="3c982f07-a1d8-4b18-a065-4e53f7e5022a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.95:15021/healthz/ready\": dial tcp 10.133.0.95:15021: connect: connection refused"
Oct 30 22:05:31.279210 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:31.279047    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-97895160-784c6ff67f-6xks4" event={"ID":"fe4f0d92-1deb-445e-b700-2e62abd9e4e3","Type":"ContainerStarted","Data":"7e5a30b9de1aa8e18968961e36c18bbc24da831ca1465aa4e3b69b1f4e3ab3ba"}
Oct 30 22:05:31.336561 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:31.280114    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-97895160-784c6ff67f-6xks4"
Oct 30 22:05:31.336561 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:31.308097    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-29cbad/premium-w6-29cbad-kserve-6c9f8d6899-qtv6c" podUID="381b11f6-3fde-4de7-9ad3-7c0891e76ba1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:05:31.336561 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:31.308351    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-29cbad/premium-w6-29cbad-kserve-6c9f8d6899-qtv6c"
Oct 30 22:05:31.347115 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:31.347071    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-0f42cb/distinct-2-w3-0f42cb-kserve-fc477897b-jk8qc" podUID="61b49182-057e-41af-b609-1a6abf82c197" containerName="main" probeResult="failure" output="Get \"https://10.133.0.80:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:05:31.347803 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:31.347778    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-0f42cb/distinct-2-w3-0f42cb-kserve-fc477897b-jk8qc"
Oct 30 22:05:31.554075 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:31.554027    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-0c99b1/distinct-2-w2-0c99b1-kserve-7466864658-8z7fk" podUID="89c09555-cf53-49a7-9c3b-95d2da2a293a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.82:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:05:31.583235 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:31.583202    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-0c99b1/distinct-2-w2-0c99b1-kserve-7466864658-8z7fk"
Oct 30 22:05:31.846404 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:31.846354    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-dde1e8/distinct-2-w0-dde1e8-kserve-744857d57-ftt8b" podUID="577b020d-2a6b-4020-863c-cf19d4f0ca3b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.79:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:05:31.847644 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:31.847604    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-dde1e8/distinct-2-w0-dde1e8-kserve-744857d57-ftt8b"
Oct 30 22:05:31.957636 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:31.957581    4473 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-0c99b1/distinct-2-w2-0c99b1-kserve-7466864658-8z7fk" podUID="89c09555-cf53-49a7-9c3b-95d2da2a293a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.82:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:05:32.139945 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:32.126060    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-0f42cb/distinct-w3-0f42cb-kserve-fb54fdd59-n6w4l" podUID="ff3c2b25-08aa-4b70-bf57-9cf4a11d05ed" containerName="main" probeResult="failure" output="Get \"https://10.133.0.73:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:05:32.139945 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:32.126473    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-9f8aea/distinct-w5-9f8aea-kserve-5b564579cd-kmxzz" podUID="b11e8b11-334f-4a9e-98c2-b0164567a8af" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": context deadline exceeded"
Oct 30 22:05:32.197906 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:32.195297    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-dde1e8/distinct-w0-dde1e8-kserve-75b68b4c5d-hts2k" podUID="9d72e7c9-657e-4278-8ed9-53adb75bd530" containerName="main" probeResult="failure" output="Get \"https://10.133.0.72:8000/ready\": context deadline exceeded"
Oct 30 22:05:32.197906 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:32.196904    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-dde1e8/distinct-w0-dde1e8-kserve-75b68b4c5d-hts2k"
Oct 30 22:05:32.197906 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:32.197832    4473 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-0f42cb/distinct-2-w3-0f42cb-kserve-fc477897b-jk8qc" podUID="61b49182-057e-41af-b609-1a6abf82c197" containerName="main" probeResult="failure" output="Get \"https://10.133.0.80:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:05:32.260421 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:32.260244    4473 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-0f42cb/distinct-w3-0f42cb-kserve-fb54fdd59-n6w4l" podUID="ff3c2b25-08aa-4b70-bf57-9cf4a11d05ed" containerName="main" probeResult="failure" output="Get \"https://10.133.0.73:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:05:32.280446 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:32.280384    4473 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-97895160-784c6ff67f-6xks4" podStartSLOduration=5.28036504 podStartE2EDuration="5.28036504s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:05:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:05:32.279649244 +0000 UTC m=+1378.053017733" watchObservedRunningTime="2026-10-30 22:05:32.28036504 +0000 UTC m=+1378.053733528"
Oct 30 22:05:32.365389 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:32.365351    4473 patch_prober.go:29] interesting pod/e2e-worker-w2-0c99b1-openshift-default-967855977-gsc42 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.95:15021/healthz/ready\": dial tcp 10.133.0.95:15021: connect: connection refused" start-of-body=
Oct 30 22:05:32.366002 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:32.365956    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-gsc42" podUID="3c982f07-a1d8-4b18-a065-4e53f7e5022a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.95:15021/healthz/ready\": dial tcp 10.133.0.95:15021: connect: connection refused"
Oct 30 22:05:32.418994 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:32.410758    4473 patch_prober.go:29] interesting pod/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-jvp7n container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 22:05:32.418994 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:32.410844    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-jvp7n" podUID="091485c0-17be-49be-a5bc-a26a9b2e89a3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:05:32.418994 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:32.411977    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-0f42cb/distinct-2-w3-0f42cb-kserve-fc477897b-jk8qc" podUID="61b49182-057e-41af-b609-1a6abf82c197" containerName="main" probeResult="failure" output="Get \"https://10.133.0.80:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:05:32.486946 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:32.486320    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-29cbad/premium-w6-29cbad-kserve-6c9f8d6899-qtv6c" podUID="381b11f6-3fde-4de7-9ad3-7c0891e76ba1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.60:8000/ready\": context deadline exceeded"
Oct 30 22:05:32.748995 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:32.737955    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-0c99b1/distinct-2-w2-0c99b1-kserve-7466864658-8z7fk" podUID="89c09555-cf53-49a7-9c3b-95d2da2a293a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.82:8000/ready\": context deadline exceeded"
Oct 30 22:05:32.879951 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:32.876529    4473 patch_prober.go:29] interesting pod/e2e-worker-w2-0c99b1-openshift-default-967855977-jsdrm container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.77:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 22:05:32.880291 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:32.880235    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-jsdrm" podUID="f100a752-adf7-4593-b9cb-9a4b276473c4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.77:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:05:32.945999 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:32.917540    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-dde1e8/distinct-2-w0-dde1e8-kserve-744857d57-ftt8b" podUID="577b020d-2a6b-4020-863c-cf19d4f0ca3b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.79:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:05:33.194846 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:33.139085    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-9f8aea/distinct-w5-9f8aea-kserve-5b564579cd-kmxzz" podUID="b11e8b11-334f-4a9e-98c2-b0164567a8af" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:05:33.194846 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:33.139168    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-0f42cb/distinct-w3-0f42cb-kserve-fb54fdd59-n6w4l" podUID="ff3c2b25-08aa-4b70-bf57-9cf4a11d05ed" containerName="main" probeResult="failure" output="Get \"https://10.133.0.73:8000/ready\": context deadline exceeded"
Oct 30 22:05:33.203158 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:33.203103    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-dde1e8/distinct-w0-dde1e8-kserve-75b68b4c5d-hts2k" podUID="9d72e7c9-657e-4278-8ed9-53adb75bd530" containerName="main" probeResult="failure" output="Get \"https://10.133.0.72:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:05:33.441950 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:33.396507    4473 patch_prober.go:29] interesting pod/e2e-worker-w2-0c99b1-openshift-default-967855977-gsc42 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.95:15021/healthz/ready\": dial tcp 10.133.0.95:15021: connect: connection refused" start-of-body=
Oct 30 22:05:33.496034 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:33.455005    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-gsc42" podUID="3c982f07-a1d8-4b18-a065-4e53f7e5022a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.95:15021/healthz/ready\": dial tcp 10.133.0.95:15021: connect: connection refused"
Oct 30 22:05:33.496034 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:33.429121    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-0f42cb/distinct-2-w3-0f42cb-kserve-fc477897b-jk8qc" podUID="61b49182-057e-41af-b609-1a6abf82c197" containerName="main" probeResult="failure" output="Get \"https://10.133.0.80:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:05:33.496034 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:33.486839    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-29cbad/premium-w6-29cbad-kserve-6c9f8d6899-qtv6c" podUID="381b11f6-3fde-4de7-9ad3-7c0891e76ba1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:05:33.695313 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:33.695263    4473 patch_prober.go:29] interesting pod/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-chw2v container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 22:05:33.695678 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:33.695634    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-chw2v" podUID="65b0ee0a-4761-40f3-bd4d-1b20dc75e233" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:05:33.743072 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:33.743029    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-0c99b1/distinct-2-w2-0c99b1-kserve-7466864658-8z7fk" podUID="89c09555-cf53-49a7-9c3b-95d2da2a293a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.82:8000/ready\": context deadline exceeded"
Oct 30 22:05:33.824102 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:33.814891    4473 patch_prober.go:29] interesting pod/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-vncql container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.74:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 22:05:33.824102 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:33.814984    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-vncql" podUID="06f032d9-4824-42fd-a92f-4cb00284ae32" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.74:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:05:33.920385 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:33.918901    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-dde1e8/distinct-2-w0-dde1e8-kserve-744857d57-ftt8b" podUID="577b020d-2a6b-4020-863c-cf19d4f0ca3b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.79:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:05:33.922507 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:33.920696    4473 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-dde1e8/distinct-w0-dde1e8-kserve-75b68b4c5d-hts2k" podUID="9d72e7c9-657e-4278-8ed9-53adb75bd530" containerName="main" probeResult="failure" output="Get \"https://10.133.0.72:8000/health\": context deadline exceeded"
Oct 30 22:05:34.195834 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:34.195792    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-9f8aea/distinct-w5-9f8aea-kserve-5b564579cd-kmxzz" podUID="b11e8b11-334f-4a9e-98c2-b0164567a8af" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": context deadline exceeded"
Oct 30 22:05:34.196529 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:34.196496    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-0f42cb/distinct-w3-0f42cb-kserve-fb54fdd59-n6w4l" podUID="ff3c2b25-08aa-4b70-bf57-9cf4a11d05ed" containerName="main" probeResult="failure" output="Get \"https://10.133.0.73:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:05:34.204951 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:34.203552    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-dde1e8/distinct-w0-dde1e8-kserve-75b68b4c5d-hts2k" podUID="9d72e7c9-657e-4278-8ed9-53adb75bd530" containerName="main" probeResult="failure" output="Get \"https://10.133.0.72:8000/ready\": context deadline exceeded"
Oct 30 22:05:34.276258 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:34.276134    4473 patch_prober.go:29] interesting pod/e2e-worker-w2-0c99b1-openshift-default-967855977-gsc42 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.95:15021/healthz/ready\": dial tcp 10.133.0.95:15021: connect: connection refused" start-of-body=
Oct 30 22:05:34.276258 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:34.276191    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-gsc42" podUID="3c982f07-a1d8-4b18-a065-4e53f7e5022a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.95:15021/healthz/ready\": dial tcp 10.133.0.95:15021: connect: connection refused"
Oct 30 22:05:34.616015 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:34.612467    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-0f42cb/distinct-2-w3-0f42cb-kserve-fc477897b-jk8qc" podUID="61b49182-057e-41af-b609-1a6abf82c197" containerName="main" probeResult="failure" output="Get \"https://10.133.0.80:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:05:34.628958 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:34.619072    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-29cbad/premium-w6-29cbad-kserve-6c9f8d6899-qtv6c" podUID="381b11f6-3fde-4de7-9ad3-7c0891e76ba1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.60:8000/ready\": context deadline exceeded"
Oct 30 22:05:34.962384 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:34.962275    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-dde1e8/distinct-2-w0-dde1e8-kserve-744857d57-ftt8b" podUID="577b020d-2a6b-4020-863c-cf19d4f0ca3b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.79:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:05:35.019991 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:35.019943    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-0c99b1/distinct-2-w2-0c99b1-kserve-7466864658-8z7fk" podUID="89c09555-cf53-49a7-9c3b-95d2da2a293a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.82:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:05:35.290014 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:35.289034    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-9f8aea/distinct-w5-9f8aea-kserve-5b564579cd-kmxzz" podUID="b11e8b11-334f-4a9e-98c2-b0164567a8af" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": context deadline exceeded"
Oct 30 22:05:35.304948 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:35.294265    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-0f42cb/distinct-w3-0f42cb-kserve-fb54fdd59-n6w4l" podUID="ff3c2b25-08aa-4b70-bf57-9cf4a11d05ed" containerName="main" probeResult="failure" output="Get \"https://10.133.0.73:8000/ready\": context deadline exceeded"
Oct 30 22:05:35.304948 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:35.295399    4473 patch_prober.go:29] interesting pod/e2e-worker-w2-0c99b1-openshift-default-967855977-gsc42 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.95:15021/healthz/ready\": dial tcp 10.133.0.95:15021: connect: connection refused" start-of-body=
Oct 30 22:05:35.304948 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:35.295440    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-gsc42" podUID="3c982f07-a1d8-4b18-a065-4e53f7e5022a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.95:15021/healthz/ready\": dial tcp 10.133.0.95:15021: connect: connection refused"
Oct 30 22:05:35.353949 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:35.352897    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-dde1e8/distinct-w0-dde1e8-kserve-75b68b4c5d-hts2k" podUID="9d72e7c9-657e-4278-8ed9-53adb75bd530" containerName="main" probeResult="failure" output="Get \"https://10.133.0.72:8000/ready\": context deadline exceeded"
Oct 30 22:05:35.475949 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:35.461043    4473 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-dde1e8/distinct-2-w0-dde1e8-kserve-744857d57-ftt8b" podUID="577b020d-2a6b-4020-863c-cf19d4f0ca3b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.79:8000/health\": context deadline exceeded"
Oct 30 22:05:35.626517 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:35.626466    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-0f42cb/distinct-2-w3-0f42cb-kserve-fc477897b-jk8qc" podUID="61b49182-057e-41af-b609-1a6abf82c197" containerName="main" probeResult="failure" output="Get \"https://10.133.0.80:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:05:35.627409 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:35.627379    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-29cbad/premium-w6-29cbad-kserve-6c9f8d6899-qtv6c" podUID="381b11f6-3fde-4de7-9ad3-7c0891e76ba1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.60:8000/ready\": context deadline exceeded"
Oct 30 22:05:35.984246 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:35.984149    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-dde1e8/distinct-2-w0-dde1e8-kserve-744857d57-ftt8b" podUID="577b020d-2a6b-4020-863c-cf19d4f0ca3b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.79:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:05:36.027667 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:36.027623    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-0c99b1/distinct-2-w2-0c99b1-kserve-7466864658-8z7fk" podUID="89c09555-cf53-49a7-9c3b-95d2da2a293a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.82:8000/ready\": context deadline exceeded"
Oct 30 22:05:36.082951 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:36.064047    4473 patch_prober.go:29] interesting pod/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-mrlnn container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.84:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 22:05:36.082951 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:36.064118    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-mrlnn" podUID="198415ba-1770-4667-8703-4693d90c8fe8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.84:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:05:36.119659 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:05:36.119616    4473 kubelet.go:2711] "Housekeeping took longer than expected" err="housekeeping took too long" expected="1s" actual="1.756s"
Oct 30 22:05:36.185948 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:36.126037    4473 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w5-9f8aea/distinct-w5-9f8aea-kserve-5b564579cd-kmxzz" podUID="b11e8b11-334f-4a9e-98c2-b0164567a8af" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:05:36.299159 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:36.282087    4473 patch_prober.go:29] interesting pod/e2e-worker-w2-0c99b1-openshift-default-967855977-gsc42 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.95:15021/healthz/ready\": dial tcp 10.133.0.95:15021: connect: connection refused" start-of-body=
Oct 30 22:05:36.299159 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:36.282149    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-gsc42" podUID="3c982f07-a1d8-4b18-a065-4e53f7e5022a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.95:15021/healthz/ready\": dial tcp 10.133.0.95:15021: connect: connection refused"
Oct 30 22:05:36.311125 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:36.311062    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-0f42cb/distinct-w3-0f42cb-kserve-fb54fdd59-n6w4l" podUID="ff3c2b25-08aa-4b70-bf57-9cf4a11d05ed" containerName="main" probeResult="failure" output="Get \"https://10.133.0.73:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:05:36.312271 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:36.312235    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-9f8aea/distinct-w5-9f8aea-kserve-5b564579cd-kmxzz" podUID="b11e8b11-334f-4a9e-98c2-b0164567a8af" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:05:36.485536 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:36.485401    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-dde1e8/distinct-w0-dde1e8-kserve-75b68b4c5d-hts2k" podUID="9d72e7c9-657e-4278-8ed9-53adb75bd530" containerName="main" probeResult="failure" output="Get \"https://10.133.0.72:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:05:36.674435 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:36.674380    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-29cbad/premium-w6-29cbad-kserve-6c9f8d6899-qtv6c" podUID="381b11f6-3fde-4de7-9ad3-7c0891e76ba1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:05:36.817935 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:36.817862    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-0f42cb/distinct-2-w3-0f42cb-kserve-fc477897b-jk8qc" podUID="61b49182-057e-41af-b609-1a6abf82c197" containerName="main" probeResult="failure" output="Get \"https://10.133.0.80:8000/ready\": context deadline exceeded"
Oct 30 22:05:36.860880 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:36.860845    4473 patch_prober.go:29] interesting pod/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-j5tqr container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 22:05:36.861184 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:36.861152    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-j5tqr" podUID="0f0fbc43-d98b-4b7c-872b-7978f76650ef" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:05:37.102020 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:37.078077    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-dde1e8/distinct-2-w0-dde1e8-kserve-744857d57-ftt8b" podUID="577b020d-2a6b-4020-863c-cf19d4f0ca3b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.79:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:05:37.327103 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:37.102328    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-0c99b1/distinct-2-w2-0c99b1-kserve-7466864658-8z7fk" podUID="89c09555-cf53-49a7-9c3b-95d2da2a293a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.82:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:05:37.331960 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:37.300036    4473 patch_prober.go:29] interesting pod/e2e-worker-w2-0c99b1-openshift-default-967855977-gsc42 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.95:15021/healthz/ready\": dial tcp 10.133.0.95:15021: connect: connection refused" start-of-body=
Oct 30 22:05:37.331960 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:37.329525    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-gsc42" podUID="3c982f07-a1d8-4b18-a065-4e53f7e5022a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.95:15021/healthz/ready\": dial tcp 10.133.0.95:15021: connect: connection refused"
Oct 30 22:05:37.331960 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:37.314301    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-0f42cb/distinct-w3-0f42cb-kserve-fb54fdd59-n6w4l" podUID="ff3c2b25-08aa-4b70-bf57-9cf4a11d05ed" containerName="main" probeResult="failure" output="Get \"https://10.133.0.73:8000/ready\": context deadline exceeded"
Oct 30 22:05:37.342779 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:37.314262    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-9f8aea/distinct-w5-9f8aea-kserve-5b564579cd-kmxzz" podUID="b11e8b11-334f-4a9e-98c2-b0164567a8af" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:05:37.521949 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:37.516492    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-dde1e8/distinct-w0-dde1e8-kserve-75b68b4c5d-hts2k" podUID="9d72e7c9-657e-4278-8ed9-53adb75bd530" containerName="main" probeResult="failure" output="Get \"https://10.133.0.72:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:05:37.709379 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:37.709340    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-29cbad/premium-w6-29cbad-kserve-6c9f8d6899-qtv6c" podUID="381b11f6-3fde-4de7-9ad3-7c0891e76ba1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.60:8000/ready\": context deadline exceeded"
Oct 30 22:05:37.857516 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:37.857398    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-0f42cb/distinct-2-w3-0f42cb-kserve-fc477897b-jk8qc" podUID="61b49182-057e-41af-b609-1a6abf82c197" containerName="main" probeResult="failure" output="Get \"https://10.133.0.80:8000/ready\": context deadline exceeded"
Oct 30 22:05:38.395979 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:38.373486    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-0c99b1/distinct-2-w2-0c99b1-kserve-7466864658-8z7fk" podUID="89c09555-cf53-49a7-9c3b-95d2da2a293a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.82:8000/ready\": context deadline exceeded"
Oct 30 22:05:38.395979 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:38.374778    4473 patch_prober.go:29] interesting pod/e2e-worker-w2-0c99b1-openshift-default-967855977-gsc42 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.95:15021/healthz/ready\": dial tcp 10.133.0.95:15021: connect: connection refused" start-of-body=
Oct 30 22:05:38.395979 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:38.374826    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-gsc42" podUID="3c982f07-a1d8-4b18-a065-4e53f7e5022a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.95:15021/healthz/ready\": dial tcp 10.133.0.95:15021: connect: connection refused"
Oct 30 22:05:38.395979 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:38.375359    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-dde1e8/distinct-2-w0-dde1e8-kserve-744857d57-ftt8b" podUID="577b020d-2a6b-4020-863c-cf19d4f0ca3b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.79:8000/ready\": context deadline exceeded"
Oct 30 22:05:38.395979 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:38.376043    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-9f8aea/distinct-w5-9f8aea-kserve-5b564579cd-kmxzz" podUID="b11e8b11-334f-4a9e-98c2-b0164567a8af" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:05:38.395979 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:38.376309    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-0f42cb/distinct-w3-0f42cb-kserve-fb54fdd59-n6w4l" podUID="ff3c2b25-08aa-4b70-bf57-9cf4a11d05ed" containerName="main" probeResult="failure" output="Get \"https://10.133.0.73:8000/ready\": context deadline exceeded"
Oct 30 22:05:38.534122 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:38.534068    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-dde1e8/distinct-w0-dde1e8-kserve-75b68b4c5d-hts2k" podUID="9d72e7c9-657e-4278-8ed9-53adb75bd530" containerName="main" probeResult="failure" output="Get \"https://10.133.0.72:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:05:38.736214 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:38.736103    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-29cbad/premium-w6-29cbad-kserve-6c9f8d6899-qtv6c" podUID="381b11f6-3fde-4de7-9ad3-7c0891e76ba1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.60:8000/ready\": context deadline exceeded"
Oct 30 22:05:38.843950 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:38.822121    4473 patch_prober.go:29] interesting pod/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-l8zrp container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.90:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 22:05:38.843950 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:38.822189    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-l8zrp" podUID="78f06b05-32bf-46bc-bd95-2b3a8da72b5b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.90:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:05:38.932964 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:38.912022    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-0f42cb/distinct-2-w3-0f42cb-kserve-fc477897b-jk8qc" podUID="61b49182-057e-41af-b609-1a6abf82c197" containerName="main" probeResult="failure" output="Get \"https://10.133.0.80:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:05:39.349139 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:39.349103    4473 patch_prober.go:29] interesting pod/e2e-worker-w2-0c99b1-openshift-default-967855977-gsc42 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.95:15021/healthz/ready\": dial tcp 10.133.0.95:15021: connect: connection refused" start-of-body=
Oct 30 22:05:39.575066 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:39.575007    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-gsc42" podUID="3c982f07-a1d8-4b18-a065-4e53f7e5022a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.95:15021/healthz/ready\": dial tcp 10.133.0.95:15021: connect: connection refused"
Oct 30 22:05:39.614551 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:39.419713    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-dde1e8/distinct-2-w0-dde1e8-kserve-744857d57-ftt8b" podUID="577b020d-2a6b-4020-863c-cf19d4f0ca3b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.79:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:05:39.620083 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:39.419660    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-0c99b1/distinct-2-w2-0c99b1-kserve-7466864658-8z7fk" podUID="89c09555-cf53-49a7-9c3b-95d2da2a293a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.82:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:05:39.633691 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:39.578108    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-dde1e8/distinct-w0-dde1e8-kserve-75b68b4c5d-hts2k" podUID="9d72e7c9-657e-4278-8ed9-53adb75bd530" containerName="main" probeResult="failure" output="Get \"https://10.133.0.72:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:05:40.319041 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:05:40.318474    4473 kubelet.go:2711] "Housekeeping took longer than expected" err="housekeeping took too long" expected="1s" actual="1.96s"
Oct 30 22:05:40.319762 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:40.319739    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-0f42cb/distinct-w3-0f42cb-kserve-fb54fdd59-n6w4l"
Oct 30 22:05:40.319995 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:40.319979    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-9f8aea/distinct-w5-9f8aea-kserve-5b564579cd-kmxzz"
Oct 30 22:05:40.320178 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:40.320165    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-29cbad/premium-w6-29cbad-kserve-6c9f8d6899-qtv6c"
Oct 30 22:05:40.320349 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:40.320336    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-0f42cb/distinct-2-w3-0f42cb-kserve-fc477897b-jk8qc"
Oct 30 22:05:40.320473 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:40.320461    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-gsc42"
Oct 30 22:05:40.383726 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:40.376252    4473 patch_prober.go:29] interesting pod/e2e-worker-w2-0c99b1-openshift-default-967855977-gsc42 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.95:15021/healthz/ready\": dial tcp 10.133.0.95:15021: connect: connection refused" start-of-body=
Oct 30 22:05:40.410253 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:40.410216    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-gsc42" podUID="3c982f07-a1d8-4b18-a065-4e53f7e5022a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.95:15021/healthz/ready\": dial tcp 10.133.0.95:15021: connect: connection refused"
Oct 30 22:05:40.410875 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:40.390634    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-0c99b1/distinct-2-w2-0c99b1-kserve-7466864658-8z7fk"
Oct 30 22:05:40.411188 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:40.384952    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-9f8aea/distinct-w5-9f8aea-kserve-5b564579cd-kmxzz" podUID="b11e8b11-334f-4a9e-98c2-b0164567a8af" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:05:40.431257 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:40.431212    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-0f42cb/distinct-2-w3-0f42cb-kserve-fc477897b-jk8qc" podUID="61b49182-057e-41af-b609-1a6abf82c197" containerName="main" probeResult="failure" output="Get \"https://10.133.0.80:8000/ready\": context deadline exceeded"
Oct 30 22:05:40.479229 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:40.479199    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-dde1e8/distinct-w0-dde1e8-kserve-75b68b4c5d-hts2k"
Oct 30 22:05:40.479519 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:40.479502    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-dde1e8/distinct-2-w0-dde1e8-kserve-744857d57-ftt8b"
Oct 30 22:05:41.301938 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:41.301884    4473 patch_prober.go:29] interesting pod/e2e-worker-w2-0c99b1-openshift-default-967855977-gsc42 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.95:15021/healthz/ready\": dial tcp 10.133.0.95:15021: connect: connection refused" start-of-body=
Oct 30 22:05:41.302222 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:41.302191    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-gsc42" podUID="3c982f07-a1d8-4b18-a065-4e53f7e5022a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.95:15021/healthz/ready\": dial tcp 10.133.0.95:15021: connect: connection refused"
Oct 30 22:05:42.334966 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:42.334904    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-gsc42"
Oct 30 22:05:42.336456 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:42.336434    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-gsc42"
Oct 30 22:05:44.327057 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:44.327032    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-97895160-784c6ff67f-6xks4"
Oct 30 22:05:50.325004 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:05:50.324876    4473 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fncvg" containerName="extractor"
Oct 30 22:05:51.162488 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:51.162451    4473 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-kp5pl"]
Oct 30 22:05:51.169262 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:51.169237    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-kp5pl"
Oct 30 22:05:51.212929 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:51.212879    4473 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-zgjtb"]
Oct 30 22:05:51.217377 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:51.217356    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-zgjtb"
Oct 30 22:05:51.227829 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:51.227805    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-875pd\" (UniqueName: \"kubernetes.io/projected/2f52b081-50a2-4489-9041-8ef19b475dba-kube-api-access-875pd\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-kp5pl\" (UID: \"2f52b081-50a2-4489-9041-8ef19b475dba\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-kp5pl"
Oct 30 22:05:51.227985 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:51.227953    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f52b081-50a2-4489-9041-8ef19b475dba-credential-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-kp5pl\" (UID: \"2f52b081-50a2-4489-9041-8ef19b475dba\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-kp5pl"
Oct 30 22:05:51.228038 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:51.228002    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2f52b081-50a2-4489-9041-8ef19b475dba-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-kp5pl\" (UID: \"2f52b081-50a2-4489-9041-8ef19b475dba\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-kp5pl"
Oct 30 22:05:51.228175 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:51.228158    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2f52b081-50a2-4489-9041-8ef19b475dba-istio-data\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-kp5pl\" (UID: \"2f52b081-50a2-4489-9041-8ef19b475dba\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-kp5pl"
Oct 30 22:05:51.228264 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:51.228249    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2f52b081-50a2-4489-9041-8ef19b475dba-istio-envoy\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-kp5pl\" (UID: \"2f52b081-50a2-4489-9041-8ef19b475dba\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-kp5pl"
Oct 30 22:05:51.228314 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:51.228281    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2f52b081-50a2-4489-9041-8ef19b475dba-istio-token\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-kp5pl\" (UID: \"2f52b081-50a2-4489-9041-8ef19b475dba\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-kp5pl"
Oct 30 22:05:51.228371 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:51.228317    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2f52b081-50a2-4489-9041-8ef19b475dba-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-kp5pl\" (UID: \"2f52b081-50a2-4489-9041-8ef19b475dba\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-kp5pl"
Oct 30 22:05:51.228433 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:51.228417    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f52b081-50a2-4489-9041-8ef19b475dba-workload-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-kp5pl\" (UID: \"2f52b081-50a2-4489-9041-8ef19b475dba\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-kp5pl"
Oct 30 22:05:51.228482 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:51.228469    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2f52b081-50a2-4489-9041-8ef19b475dba-workload-certs\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-kp5pl\" (UID: \"2f52b081-50a2-4489-9041-8ef19b475dba\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-kp5pl"
Oct 30 22:05:51.286860 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:51.286823    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-kp5pl"]
Oct 30 22:05:51.306074 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:51.306037    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-zgjtb"]
Oct 30 22:05:51.329767 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:51.329732    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2f52b081-50a2-4489-9041-8ef19b475dba-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-kp5pl\" (UID: \"2f52b081-50a2-4489-9041-8ef19b475dba\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-kp5pl"
Oct 30 22:05:51.330478 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:51.329801    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a13c1563-5b04-424a-b761-62a4b85f16e7-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-zgjtb\" (UID: \"a13c1563-5b04-424a-b761-62a4b85f16e7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-zgjtb"
Oct 30 22:05:51.330478 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:51.329866    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9n2kk\" (UniqueName: \"kubernetes.io/projected/a13c1563-5b04-424a-b761-62a4b85f16e7-kube-api-access-9n2kk\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-zgjtb\" (UID: \"a13c1563-5b04-424a-b761-62a4b85f16e7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-zgjtb"
Oct 30 22:05:51.330478 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:51.329982    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a13c1563-5b04-424a-b761-62a4b85f16e7-workload-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-zgjtb\" (UID: \"a13c1563-5b04-424a-b761-62a4b85f16e7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-zgjtb"
Oct 30 22:05:51.330478 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:51.330017    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a13c1563-5b04-424a-b761-62a4b85f16e7-istio-data\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-zgjtb\" (UID: \"a13c1563-5b04-424a-b761-62a4b85f16e7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-zgjtb"
Oct 30 22:05:51.330478 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:51.330134    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2f52b081-50a2-4489-9041-8ef19b475dba-istio-data\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-kp5pl\" (UID: \"2f52b081-50a2-4489-9041-8ef19b475dba\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-kp5pl"
Oct 30 22:05:51.330478 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:51.330165    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a13c1563-5b04-424a-b761-62a4b85f16e7-credential-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-zgjtb\" (UID: \"a13c1563-5b04-424a-b761-62a4b85f16e7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-zgjtb"
Oct 30 22:05:51.330478 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:51.330269    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a13c1563-5b04-424a-b761-62a4b85f16e7-workload-certs\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-zgjtb\" (UID: \"a13c1563-5b04-424a-b761-62a4b85f16e7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-zgjtb"
Oct 30 22:05:51.330478 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:51.330304    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a13c1563-5b04-424a-b761-62a4b85f16e7-istio-token\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-zgjtb\" (UID: \"a13c1563-5b04-424a-b761-62a4b85f16e7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-zgjtb"
Oct 30 22:05:51.330478 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:51.330332    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2f52b081-50a2-4489-9041-8ef19b475dba-istio-envoy\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-kp5pl\" (UID: \"2f52b081-50a2-4489-9041-8ef19b475dba\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-kp5pl"
Oct 30 22:05:51.330478 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:51.330355    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2f52b081-50a2-4489-9041-8ef19b475dba-istio-token\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-kp5pl\" (UID: \"2f52b081-50a2-4489-9041-8ef19b475dba\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-kp5pl"
Oct 30 22:05:51.330478 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:51.330405    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2f52b081-50a2-4489-9041-8ef19b475dba-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-kp5pl\" (UID: \"2f52b081-50a2-4489-9041-8ef19b475dba\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-kp5pl"
Oct 30 22:05:51.331104 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:51.330499    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f52b081-50a2-4489-9041-8ef19b475dba-workload-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-kp5pl\" (UID: \"2f52b081-50a2-4489-9041-8ef19b475dba\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-kp5pl"
Oct 30 22:05:51.331104 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:51.330529    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2f52b081-50a2-4489-9041-8ef19b475dba-istio-data\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-kp5pl\" (UID: \"2f52b081-50a2-4489-9041-8ef19b475dba\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-kp5pl"
Oct 30 22:05:51.331104 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:51.330551    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2f52b081-50a2-4489-9041-8ef19b475dba-workload-certs\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-kp5pl\" (UID: \"2f52b081-50a2-4489-9041-8ef19b475dba\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-kp5pl"
Oct 30 22:05:51.331104 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:51.330576    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a13c1563-5b04-424a-b761-62a4b85f16e7-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-zgjtb\" (UID: \"a13c1563-5b04-424a-b761-62a4b85f16e7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-zgjtb"
Oct 30 22:05:51.331104 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:51.330587    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2f52b081-50a2-4489-9041-8ef19b475dba-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-kp5pl\" (UID: \"2f52b081-50a2-4489-9041-8ef19b475dba\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-kp5pl"
Oct 30 22:05:51.331104 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:51.330618    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-875pd\" (UniqueName: \"kubernetes.io/projected/2f52b081-50a2-4489-9041-8ef19b475dba-kube-api-access-875pd\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-kp5pl\" (UID: \"2f52b081-50a2-4489-9041-8ef19b475dba\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-kp5pl"
Oct 30 22:05:51.331104 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:51.330671    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a13c1563-5b04-424a-b761-62a4b85f16e7-istio-envoy\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-zgjtb\" (UID: \"a13c1563-5b04-424a-b761-62a4b85f16e7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-zgjtb"
Oct 30 22:05:51.331104 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:51.330793    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f52b081-50a2-4489-9041-8ef19b475dba-credential-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-kp5pl\" (UID: \"2f52b081-50a2-4489-9041-8ef19b475dba\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-kp5pl"
Oct 30 22:05:51.331346 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:51.331143    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f52b081-50a2-4489-9041-8ef19b475dba-workload-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-kp5pl\" (UID: \"2f52b081-50a2-4489-9041-8ef19b475dba\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-kp5pl"
Oct 30 22:05:51.331755 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:51.331725    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f52b081-50a2-4489-9041-8ef19b475dba-credential-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-kp5pl\" (UID: \"2f52b081-50a2-4489-9041-8ef19b475dba\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-kp5pl"
Oct 30 22:05:51.331961 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:51.331941    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2f52b081-50a2-4489-9041-8ef19b475dba-workload-certs\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-kp5pl\" (UID: \"2f52b081-50a2-4489-9041-8ef19b475dba\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-kp5pl"
Oct 30 22:05:51.333582 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:51.333556    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2f52b081-50a2-4489-9041-8ef19b475dba-istio-envoy\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-kp5pl\" (UID: \"2f52b081-50a2-4489-9041-8ef19b475dba\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-kp5pl"
Oct 30 22:05:51.339488 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:51.339463    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2f52b081-50a2-4489-9041-8ef19b475dba-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-kp5pl\" (UID: \"2f52b081-50a2-4489-9041-8ef19b475dba\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-kp5pl"
Oct 30 22:05:51.388638 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:51.388604    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-875pd\" (UniqueName: \"kubernetes.io/projected/2f52b081-50a2-4489-9041-8ef19b475dba-kube-api-access-875pd\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-kp5pl\" (UID: \"2f52b081-50a2-4489-9041-8ef19b475dba\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-kp5pl"
Oct 30 22:05:51.388638 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:51.388627    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2f52b081-50a2-4489-9041-8ef19b475dba-istio-token\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-kp5pl\" (UID: \"2f52b081-50a2-4489-9041-8ef19b475dba\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-kp5pl"
Oct 30 22:05:51.431892 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:51.431796    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a13c1563-5b04-424a-b761-62a4b85f16e7-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-zgjtb\" (UID: \"a13c1563-5b04-424a-b761-62a4b85f16e7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-zgjtb"
Oct 30 22:05:51.431892 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:51.431880    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9n2kk\" (UniqueName: \"kubernetes.io/projected/a13c1563-5b04-424a-b761-62a4b85f16e7-kube-api-access-9n2kk\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-zgjtb\" (UID: \"a13c1563-5b04-424a-b761-62a4b85f16e7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-zgjtb"
Oct 30 22:05:51.432189 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:51.431909    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a13c1563-5b04-424a-b761-62a4b85f16e7-workload-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-zgjtb\" (UID: \"a13c1563-5b04-424a-b761-62a4b85f16e7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-zgjtb"
Oct 30 22:05:51.432189 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:51.431948    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a13c1563-5b04-424a-b761-62a4b85f16e7-istio-data\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-zgjtb\" (UID: \"a13c1563-5b04-424a-b761-62a4b85f16e7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-zgjtb"
Oct 30 22:05:51.432189 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:51.432024    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a13c1563-5b04-424a-b761-62a4b85f16e7-credential-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-zgjtb\" (UID: \"a13c1563-5b04-424a-b761-62a4b85f16e7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-zgjtb"
Oct 30 22:05:51.432189 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:51.432157    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a13c1563-5b04-424a-b761-62a4b85f16e7-workload-certs\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-zgjtb\" (UID: \"a13c1563-5b04-424a-b761-62a4b85f16e7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-zgjtb"
Oct 30 22:05:51.432396 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:51.432203    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a13c1563-5b04-424a-b761-62a4b85f16e7-istio-token\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-zgjtb\" (UID: \"a13c1563-5b04-424a-b761-62a4b85f16e7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-zgjtb"
Oct 30 22:05:51.432396 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:51.432390    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a13c1563-5b04-424a-b761-62a4b85f16e7-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-zgjtb\" (UID: \"a13c1563-5b04-424a-b761-62a4b85f16e7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-zgjtb"
Oct 30 22:05:51.432510 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:51.432410    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a13c1563-5b04-424a-b761-62a4b85f16e7-credential-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-zgjtb\" (UID: \"a13c1563-5b04-424a-b761-62a4b85f16e7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-zgjtb"
Oct 30 22:05:51.432510 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:51.432448    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a13c1563-5b04-424a-b761-62a4b85f16e7-istio-envoy\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-zgjtb\" (UID: \"a13c1563-5b04-424a-b761-62a4b85f16e7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-zgjtb"
Oct 30 22:05:51.432510 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:51.432471    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a13c1563-5b04-424a-b761-62a4b85f16e7-workload-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-zgjtb\" (UID: \"a13c1563-5b04-424a-b761-62a4b85f16e7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-zgjtb"
Oct 30 22:05:51.432688 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:51.432533    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a13c1563-5b04-424a-b761-62a4b85f16e7-istio-data\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-zgjtb\" (UID: \"a13c1563-5b04-424a-b761-62a4b85f16e7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-zgjtb"
Oct 30 22:05:51.432776 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:51.432681    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a13c1563-5b04-424a-b761-62a4b85f16e7-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-zgjtb\" (UID: \"a13c1563-5b04-424a-b761-62a4b85f16e7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-zgjtb"
Oct 30 22:05:51.432776 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:51.432707    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a13c1563-5b04-424a-b761-62a4b85f16e7-workload-certs\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-zgjtb\" (UID: \"a13c1563-5b04-424a-b761-62a4b85f16e7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-zgjtb"
Oct 30 22:05:51.435065 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:51.435037    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a13c1563-5b04-424a-b761-62a4b85f16e7-istio-envoy\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-zgjtb\" (UID: \"a13c1563-5b04-424a-b761-62a4b85f16e7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-zgjtb"
Oct 30 22:05:51.435259 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:51.435238    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a13c1563-5b04-424a-b761-62a4b85f16e7-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-zgjtb\" (UID: \"a13c1563-5b04-424a-b761-62a4b85f16e7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-zgjtb"
Oct 30 22:05:51.484065 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:51.484027    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-kp5pl"
Oct 30 22:05:51.491531 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:51.491493    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a13c1563-5b04-424a-b761-62a4b85f16e7-istio-token\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-zgjtb\" (UID: \"a13c1563-5b04-424a-b761-62a4b85f16e7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-zgjtb"
Oct 30 22:05:51.491649 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:51.491581    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9n2kk\" (UniqueName: \"kubernetes.io/projected/a13c1563-5b04-424a-b761-62a4b85f16e7-kube-api-access-9n2kk\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-zgjtb\" (UID: \"a13c1563-5b04-424a-b761-62a4b85f16e7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-zgjtb"
Oct 30 22:05:51.532892 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:51.532862    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-zgjtb"
Oct 30 22:05:51.914742 ip-10-0-140-18 kubenswrapper[4473]: W1030 22:05:51.914714    4473 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f52b081_50a2_4489_9041_8ef19b475dba.slice/crio-3f595a983cb6f0f3cac82d22803a6829c2670763f91d32f737fffc9d510475ef WatchSource:0}: Error finding container 3f595a983cb6f0f3cac82d22803a6829c2670763f91d32f737fffc9d510475ef: Status 404 returned error can't find the container with id 3f595a983cb6f0f3cac82d22803a6829c2670763f91d32f737fffc9d510475ef
Oct 30 22:05:51.915551 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:51.915526    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-kp5pl"]
Oct 30 22:05:51.917905 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:51.917872    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:05:51.918050 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:51.917977    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:05:51.918050 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:51.918014    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:05:51.969018 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:51.968980    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-zgjtb"]
Oct 30 22:05:51.978255 ip-10-0-140-18 kubenswrapper[4473]: W1030 22:05:51.978228    4473 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda13c1563_5b04_424a_b761_62a4b85f16e7.slice/crio-9395406ff516a4543a0dbc244a0b1ee76ddbdf9296eba2353eb954c0c54767b0 WatchSource:0}: Error finding container 9395406ff516a4543a0dbc244a0b1ee76ddbdf9296eba2353eb954c0c54767b0: Status 404 returned error can't find the container with id 9395406ff516a4543a0dbc244a0b1ee76ddbdf9296eba2353eb954c0c54767b0
Oct 30 22:05:51.981535 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:51.981489    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:05:51.981636 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:51.981584    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:05:51.981636 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:51.981623    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:05:52.699220 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:52.699184    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-kp5pl" event={"ID":"2f52b081-50a2-4489-9041-8ef19b475dba","Type":"ContainerStarted","Data":"aa50cae6369e881f07d0457f17ba921d95856c73fabf896712efd7e86d536fae"}
Oct 30 22:05:52.699804 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:52.699243    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-kp5pl" event={"ID":"2f52b081-50a2-4489-9041-8ef19b475dba","Type":"ContainerStarted","Data":"3f595a983cb6f0f3cac82d22803a6829c2670763f91d32f737fffc9d510475ef"}
Oct 30 22:05:52.700770 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:52.700742    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-zgjtb" event={"ID":"a13c1563-5b04-424a-b761-62a4b85f16e7","Type":"ContainerStarted","Data":"b1d49a3aa2d6a87165edc0e4ad00fec72249d455f04a9b3dec6d75f1b3678928"}
Oct 30 22:05:52.700872 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:52.700789    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-zgjtb" event={"ID":"a13c1563-5b04-424a-b761-62a4b85f16e7","Type":"ContainerStarted","Data":"9395406ff516a4543a0dbc244a0b1ee76ddbdf9296eba2353eb954c0c54767b0"}
Oct 30 22:05:52.973894 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:52.973768    4473 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-kp5pl" podStartSLOduration=1.9737467400000002 podStartE2EDuration="1.97374674s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:05:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:05:52.850794154 +0000 UTC m=+1398.624162655" watchObservedRunningTime="2026-10-30 22:05:52.97374674 +0000 UTC m=+1398.747115228"
Oct 30 22:05:52.976409 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:52.976349    4473 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-zgjtb" podStartSLOduration=1.976335111 podStartE2EDuration="1.976335111s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:05:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:05:52.967963707 +0000 UTC m=+1398.741332195" watchObservedRunningTime="2026-10-30 22:05:52.976335111 +0000 UTC m=+1398.749703599"
Oct 30 22:05:53.485048 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:53.484773    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-kp5pl"
Oct 30 22:05:53.486025 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:53.485999    4473 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-kp5pl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.97:15021/healthz/ready\": dial tcp 10.133.0.97:15021: connect: connection refused" start-of-body=
Oct 30 22:05:53.486149 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:53.486056    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-kp5pl" podUID="2f52b081-50a2-4489-9041-8ef19b475dba" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.97:15021/healthz/ready\": dial tcp 10.133.0.97:15021: connect: connection refused"
Oct 30 22:05:53.547945 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:53.546787    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-zgjtb"
Oct 30 22:05:53.568948 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:53.552048    4473 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-zgjtb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.98:15021/healthz/ready\": dial tcp 10.133.0.98:15021: connect: connection refused" start-of-body=
Oct 30 22:05:53.568948 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:53.552114    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-zgjtb" podUID="a13c1563-5b04-424a-b761-62a4b85f16e7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.98:15021/healthz/ready\": dial tcp 10.133.0.98:15021: connect: connection refused"
Oct 30 22:05:54.484481 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:54.484429    4473 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-kp5pl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.97:15021/healthz/ready\": dial tcp 10.133.0.97:15021: connect: connection refused" start-of-body=
Oct 30 22:05:54.484974 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:54.484504    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-kp5pl" podUID="2f52b081-50a2-4489-9041-8ef19b475dba" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.97:15021/healthz/ready\": dial tcp 10.133.0.97:15021: connect: connection refused"
Oct 30 22:05:54.536946 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:54.534363    4473 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-zgjtb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.98:15021/healthz/ready\": dial tcp 10.133.0.98:15021: connect: connection refused" start-of-body=
Oct 30 22:05:54.536946 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:54.534424    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-zgjtb" podUID="a13c1563-5b04-424a-b761-62a4b85f16e7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.98:15021/healthz/ready\": dial tcp 10.133.0.98:15021: connect: connection refused"
Oct 30 22:05:55.484609 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:55.484562    4473 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-kp5pl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.97:15021/healthz/ready\": dial tcp 10.133.0.97:15021: connect: connection refused" start-of-body=
Oct 30 22:05:55.485042 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:55.484627    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-kp5pl" podUID="2f52b081-50a2-4489-9041-8ef19b475dba" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.97:15021/healthz/ready\": dial tcp 10.133.0.97:15021: connect: connection refused"
Oct 30 22:05:55.541947 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:55.533360    4473 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-zgjtb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.98:15021/healthz/ready\": dial tcp 10.133.0.98:15021: connect: connection refused" start-of-body=
Oct 30 22:05:55.541947 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:55.533432    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-zgjtb" podUID="a13c1563-5b04-424a-b761-62a4b85f16e7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.98:15021/healthz/ready\": dial tcp 10.133.0.98:15021: connect: connection refused"
Oct 30 22:05:56.485207 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:56.485177    4473 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-kp5pl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.97:15021/healthz/ready\": dial tcp 10.133.0.97:15021: connect: connection refused" start-of-body=
Oct 30 22:05:56.485607 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:56.485236    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-kp5pl" podUID="2f52b081-50a2-4489-9041-8ef19b475dba" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.97:15021/healthz/ready\": dial tcp 10.133.0.97:15021: connect: connection refused"
Oct 30 22:05:56.535151 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:56.535093    4473 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-zgjtb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.98:15021/healthz/ready\": dial tcp 10.133.0.98:15021: connect: connection refused" start-of-body=
Oct 30 22:05:56.535454 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:56.535426    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-zgjtb" podUID="a13c1563-5b04-424a-b761-62a4b85f16e7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.98:15021/healthz/ready\": dial tcp 10.133.0.98:15021: connect: connection refused"
Oct 30 22:05:57.484844 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:57.484805    4473 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-kp5pl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.97:15021/healthz/ready\": dial tcp 10.133.0.97:15021: connect: connection refused" start-of-body=
Oct 30 22:05:57.485077 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:57.484892    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-kp5pl" podUID="2f52b081-50a2-4489-9041-8ef19b475dba" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.97:15021/healthz/ready\": dial tcp 10.133.0.97:15021: connect: connection refused"
Oct 30 22:05:57.544947 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:57.544382    4473 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-zgjtb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.98:15021/healthz/ready\": dial tcp 10.133.0.98:15021: connect: connection refused" start-of-body=
Oct 30 22:05:57.544947 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:57.544449    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-zgjtb" podUID="a13c1563-5b04-424a-b761-62a4b85f16e7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.98:15021/healthz/ready\": dial tcp 10.133.0.98:15021: connect: connection refused"
Oct 30 22:05:58.485507 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:58.485471    4473 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-kp5pl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.97:15021/healthz/ready\": dial tcp 10.133.0.97:15021: connect: connection refused" start-of-body=
Oct 30 22:05:58.485734 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:58.485550    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-kp5pl" podUID="2f52b081-50a2-4489-9041-8ef19b475dba" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.97:15021/healthz/ready\": dial tcp 10.133.0.97:15021: connect: connection refused"
Oct 30 22:05:58.533393 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:58.533355    4473 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-zgjtb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.98:15021/healthz/ready\": dial tcp 10.133.0.98:15021: connect: connection refused" start-of-body=
Oct 30 22:05:58.533574 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:58.533435    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-zgjtb" podUID="a13c1563-5b04-424a-b761-62a4b85f16e7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.98:15021/healthz/ready\": dial tcp 10.133.0.98:15021: connect: connection refused"
Oct 30 22:05:59.485081 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:59.484953    4473 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-kp5pl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.97:15021/healthz/ready\": dial tcp 10.133.0.97:15021: connect: connection refused" start-of-body=
Oct 30 22:05:59.485616 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:59.485121    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-kp5pl" podUID="2f52b081-50a2-4489-9041-8ef19b475dba" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.97:15021/healthz/ready\": dial tcp 10.133.0.97:15021: connect: connection refused"
Oct 30 22:05:59.533963 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:59.533898    4473 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-zgjtb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.98:15021/healthz/ready\": dial tcp 10.133.0.98:15021: connect: connection refused" start-of-body=
Oct 30 22:05:59.534158 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:05:59.534010    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-zgjtb" podUID="a13c1563-5b04-424a-b761-62a4b85f16e7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.98:15021/healthz/ready\": dial tcp 10.133.0.98:15021: connect: connection refused"
Oct 30 22:06:00.630848 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:06:00.630797    4473 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-1d71f464-7b656bc7b4-pqlb6"]
Oct 30 22:06:00.637515 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:06:00.637478    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1d71f464-7b656bc7b4-pqlb6"
Oct 30 22:06:00.690441 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:06:00.690412    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-1d71f464-7b656bc7b4-pqlb6"]
Oct 30 22:06:00.698454 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:06:00.698428    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-kp5pl"
Oct 30 22:06:00.699554 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:06:00.699517    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-kp5pl"
Oct 30 22:06:00.701037 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:06:00.701015    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-kp5pl"
Oct 30 22:06:00.734168 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:06:00.734132    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-1d71f464-serving-cert\""
Oct 30 22:06:00.762711 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:06:00.762667    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n7bdw\" (UniqueName: \"kubernetes.io/projected/976cc05a-058b-43d0-95f6-75e6f0425e2f-kube-api-access-n7bdw\") pod \"maas-api-e2e-ait-1d71f464-7b656bc7b4-pqlb6\" (UID: \"976cc05a-058b-43d0-95f6-75e6f0425e2f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1d71f464-7b656bc7b4-pqlb6"
Oct 30 22:06:00.762955 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:06:00.762775    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/976cc05a-058b-43d0-95f6-75e6f0425e2f-metrics-certs\") pod \"maas-api-e2e-ait-1d71f464-7b656bc7b4-pqlb6\" (UID: \"976cc05a-058b-43d0-95f6-75e6f0425e2f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1d71f464-7b656bc7b4-pqlb6"
Oct 30 22:06:00.763362 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:06:00.763335    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/976cc05a-058b-43d0-95f6-75e6f0425e2f-openshift-service-ca\") pod \"maas-api-e2e-ait-1d71f464-7b656bc7b4-pqlb6\" (UID: \"976cc05a-058b-43d0-95f6-75e6f0425e2f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1d71f464-7b656bc7b4-pqlb6"
Oct 30 22:06:00.763533 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:06:00.763513    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/976cc05a-058b-43d0-95f6-75e6f0425e2f-maas-api-tls\") pod \"maas-api-e2e-ait-1d71f464-7b656bc7b4-pqlb6\" (UID: \"976cc05a-058b-43d0-95f6-75e6f0425e2f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1d71f464-7b656bc7b4-pqlb6"
Oct 30 22:06:00.793599 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:06:00.793566    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-zgjtb"
Oct 30 22:06:00.794779 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:06:00.794756    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-zgjtb"
Oct 30 22:06:00.795975 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:06:00.795945    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-zgjtb"
Oct 30 22:06:00.865369 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:06:00.865315    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/976cc05a-058b-43d0-95f6-75e6f0425e2f-openshift-service-ca\") pod \"maas-api-e2e-ait-1d71f464-7b656bc7b4-pqlb6\" (UID: \"976cc05a-058b-43d0-95f6-75e6f0425e2f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1d71f464-7b656bc7b4-pqlb6"
Oct 30 22:06:00.865583 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:06:00.865393    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/976cc05a-058b-43d0-95f6-75e6f0425e2f-maas-api-tls\") pod \"maas-api-e2e-ait-1d71f464-7b656bc7b4-pqlb6\" (UID: \"976cc05a-058b-43d0-95f6-75e6f0425e2f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1d71f464-7b656bc7b4-pqlb6"
Oct 30 22:06:00.865583 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:06:00.865432    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n7bdw\" (UniqueName: \"kubernetes.io/projected/976cc05a-058b-43d0-95f6-75e6f0425e2f-kube-api-access-n7bdw\") pod \"maas-api-e2e-ait-1d71f464-7b656bc7b4-pqlb6\" (UID: \"976cc05a-058b-43d0-95f6-75e6f0425e2f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1d71f464-7b656bc7b4-pqlb6"
Oct 30 22:06:00.865583 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:06:00.865505    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/976cc05a-058b-43d0-95f6-75e6f0425e2f-metrics-certs\") pod \"maas-api-e2e-ait-1d71f464-7b656bc7b4-pqlb6\" (UID: \"976cc05a-058b-43d0-95f6-75e6f0425e2f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1d71f464-7b656bc7b4-pqlb6"
Oct 30 22:06:00.866027 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:06:00.865949    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/976cc05a-058b-43d0-95f6-75e6f0425e2f-openshift-service-ca\") pod \"maas-api-e2e-ait-1d71f464-7b656bc7b4-pqlb6\" (UID: \"976cc05a-058b-43d0-95f6-75e6f0425e2f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1d71f464-7b656bc7b4-pqlb6"
Oct 30 22:06:00.866297 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:06:00.866275    4473 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-1d71f464-serving-cert: secret "maas-api-e2e-ait-1d71f464-serving-cert" not found
Oct 30 22:06:00.866395 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:06:00.866358    4473 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/976cc05a-058b-43d0-95f6-75e6f0425e2f-maas-api-tls podName:976cc05a-058b-43d0-95f6-75e6f0425e2f nodeName:}" failed. No retries permitted until 2026-10-30 22:06:01.36633902 +0000 UTC m=+1407.139707503 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/976cc05a-058b-43d0-95f6-75e6f0425e2f-maas-api-tls") pod "maas-api-e2e-ait-1d71f464-7b656bc7b4-pqlb6" (UID: "976cc05a-058b-43d0-95f6-75e6f0425e2f") : secret "maas-api-e2e-ait-1d71f464-serving-cert" not found
Oct 30 22:06:00.868383 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:06:00.868357    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/976cc05a-058b-43d0-95f6-75e6f0425e2f-metrics-certs\") pod \"maas-api-e2e-ait-1d71f464-7b656bc7b4-pqlb6\" (UID: \"976cc05a-058b-43d0-95f6-75e6f0425e2f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1d71f464-7b656bc7b4-pqlb6"
Oct 30 22:06:01.163821 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:06:01.163779    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n7bdw\" (UniqueName: \"kubernetes.io/projected/976cc05a-058b-43d0-95f6-75e6f0425e2f-kube-api-access-n7bdw\") pod \"maas-api-e2e-ait-1d71f464-7b656bc7b4-pqlb6\" (UID: \"976cc05a-058b-43d0-95f6-75e6f0425e2f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1d71f464-7b656bc7b4-pqlb6"
Oct 30 22:06:01.467734 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:06:01.467655    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/976cc05a-058b-43d0-95f6-75e6f0425e2f-maas-api-tls\") pod \"maas-api-e2e-ait-1d71f464-7b656bc7b4-pqlb6\" (UID: \"976cc05a-058b-43d0-95f6-75e6f0425e2f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1d71f464-7b656bc7b4-pqlb6"
Oct 30 22:06:01.470561 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:06:01.470531    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/976cc05a-058b-43d0-95f6-75e6f0425e2f-maas-api-tls\") pod \"maas-api-e2e-ait-1d71f464-7b656bc7b4-pqlb6\" (UID: \"976cc05a-058b-43d0-95f6-75e6f0425e2f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1d71f464-7b656bc7b4-pqlb6"
Oct 30 22:06:01.564348 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:06:01.564304    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1d71f464-7b656bc7b4-pqlb6"
Oct 30 22:06:01.948377 ip-10-0-140-18 kubenswrapper[4473]: W1030 22:06:01.948341    4473 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod976cc05a_058b_43d0_95f6_75e6f0425e2f.slice/crio-402401d3378953e78fc358ba38caaebea54f532d07a0a04a925b972f3781792b WatchSource:0}: Error finding container 402401d3378953e78fc358ba38caaebea54f532d07a0a04a925b972f3781792b: Status 404 returned error can't find the container with id 402401d3378953e78fc358ba38caaebea54f532d07a0a04a925b972f3781792b
Oct 30 22:06:01.949508 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:06:01.949481    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-1d71f464-7b656bc7b4-pqlb6"]
Oct 30 22:06:01.949904 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:06:01.949884    4473 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 22:06:02.828018 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:06:02.827977    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1d71f464-7b656bc7b4-pqlb6" event={"ID":"976cc05a-058b-43d0-95f6-75e6f0425e2f","Type":"ContainerStarted","Data":"402401d3378953e78fc358ba38caaebea54f532d07a0a04a925b972f3781792b"}
Oct 30 22:06:03.837697 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:06:03.837662    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1d71f464-7b656bc7b4-pqlb6" event={"ID":"976cc05a-058b-43d0-95f6-75e6f0425e2f","Type":"ContainerStarted","Data":"d6b59061e22da417c7cf5b73e528603ce9e69c902534caedb8187a6fbe0ffc6d"}
Oct 30 22:06:03.838148 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:06:03.837795    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1d71f464-7b656bc7b4-pqlb6"
Oct 30 22:06:09.850148 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:06:09.850110    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1d71f464-7b656bc7b4-pqlb6"
Oct 30 22:06:10.017391 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:06:10.017338    4473 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1d71f464-7b656bc7b4-pqlb6" podStartSLOduration=9.119742986 podStartE2EDuration="10.01732193s" totalImagesPullingTime="897.578944ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:06:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:06:03.962239212 +0000 UTC m=+1409.735607700" watchObservedRunningTime="2026-10-30 22:06:10.01732193 +0000 UTC m=+1415.790690416"
Oct 30 22:06:10.697520 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:06:10.697483    4473 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-64086213-799ffc7fbc-jjd6r"]
Oct 30 22:06:10.703084 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:06:10.703058    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-64086213-799ffc7fbc-jjd6r"
Oct 30 22:06:10.760977 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:06:10.760938    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-64086213-799ffc7fbc-jjd6r"]
Oct 30 22:06:10.805865 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:06:10.805832    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-64086213-serving-cert\""
Oct 30 22:06:10.828638 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:06:10.828603    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8c51747e-187c-403e-8bfb-98b13d0d9077-metrics-certs\") pod \"maas-api-e2e-ait-64086213-799ffc7fbc-jjd6r\" (UID: \"8c51747e-187c-403e-8bfb-98b13d0d9077\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-64086213-799ffc7fbc-jjd6r"
Oct 30 22:06:10.828846 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:06:10.828657    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cfxlc\" (UniqueName: \"kubernetes.io/projected/8c51747e-187c-403e-8bfb-98b13d0d9077-kube-api-access-cfxlc\") pod \"maas-api-e2e-ait-64086213-799ffc7fbc-jjd6r\" (UID: \"8c51747e-187c-403e-8bfb-98b13d0d9077\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-64086213-799ffc7fbc-jjd6r"
Oct 30 22:06:10.828986 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:06:10.828970    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8c51747e-187c-403e-8bfb-98b13d0d9077-maas-api-tls\") pod \"maas-api-e2e-ait-64086213-799ffc7fbc-jjd6r\" (UID: \"8c51747e-187c-403e-8bfb-98b13d0d9077\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-64086213-799ffc7fbc-jjd6r"
Oct 30 22:06:10.829046 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:06:10.829034    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8c51747e-187c-403e-8bfb-98b13d0d9077-openshift-service-ca\") pod \"maas-api-e2e-ait-64086213-799ffc7fbc-jjd6r\" (UID: \"8c51747e-187c-403e-8bfb-98b13d0d9077\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-64086213-799ffc7fbc-jjd6r"
Oct 30 22:06:10.930818 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:06:10.930778    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8c51747e-187c-403e-8bfb-98b13d0d9077-metrics-certs\") pod \"maas-api-e2e-ait-64086213-799ffc7fbc-jjd6r\" (UID: \"8c51747e-187c-403e-8bfb-98b13d0d9077\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-64086213-799ffc7fbc-jjd6r"
Oct 30 22:06:10.931361 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:06:10.930833    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cfxlc\" (UniqueName: \"kubernetes.io/projected/8c51747e-187c-403e-8bfb-98b13d0d9077-kube-api-access-cfxlc\") pod \"maas-api-e2e-ait-64086213-799ffc7fbc-jjd6r\" (UID: \"8c51747e-187c-403e-8bfb-98b13d0d9077\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-64086213-799ffc7fbc-jjd6r"
Oct 30 22:06:10.931361 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:06:10.931129    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8c51747e-187c-403e-8bfb-98b13d0d9077-maas-api-tls\") pod \"maas-api-e2e-ait-64086213-799ffc7fbc-jjd6r\" (UID: \"8c51747e-187c-403e-8bfb-98b13d0d9077\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-64086213-799ffc7fbc-jjd6r"
Oct 30 22:06:10.931361 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:06:10.931210    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8c51747e-187c-403e-8bfb-98b13d0d9077-openshift-service-ca\") pod \"maas-api-e2e-ait-64086213-799ffc7fbc-jjd6r\" (UID: \"8c51747e-187c-403e-8bfb-98b13d0d9077\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-64086213-799ffc7fbc-jjd6r"
Oct 30 22:06:10.931361 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:06:10.931329    4473 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-64086213-serving-cert: secret "maas-api-e2e-ait-64086213-serving-cert" not found
Oct 30 22:06:10.931610 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:06:10.931433    4473 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8c51747e-187c-403e-8bfb-98b13d0d9077-maas-api-tls podName:8c51747e-187c-403e-8bfb-98b13d0d9077 nodeName:}" failed. No retries permitted until 2026-10-30 22:06:11.431410501 +0000 UTC m=+1417.204778979 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/8c51747e-187c-403e-8bfb-98b13d0d9077-maas-api-tls") pod "maas-api-e2e-ait-64086213-799ffc7fbc-jjd6r" (UID: "8c51747e-187c-403e-8bfb-98b13d0d9077") : secret "maas-api-e2e-ait-64086213-serving-cert" not found
Oct 30 22:06:10.931993 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:06:10.931971    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8c51747e-187c-403e-8bfb-98b13d0d9077-openshift-service-ca\") pod \"maas-api-e2e-ait-64086213-799ffc7fbc-jjd6r\" (UID: \"8c51747e-187c-403e-8bfb-98b13d0d9077\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-64086213-799ffc7fbc-jjd6r"
Oct 30 22:06:10.933731 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:06:10.933704    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8c51747e-187c-403e-8bfb-98b13d0d9077-metrics-certs\") pod \"maas-api-e2e-ait-64086213-799ffc7fbc-jjd6r\" (UID: \"8c51747e-187c-403e-8bfb-98b13d0d9077\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-64086213-799ffc7fbc-jjd6r"
Oct 30 22:06:11.034111 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:06:11.031216    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cfxlc\" (UniqueName: \"kubernetes.io/projected/8c51747e-187c-403e-8bfb-98b13d0d9077-kube-api-access-cfxlc\") pod \"maas-api-e2e-ait-64086213-799ffc7fbc-jjd6r\" (UID: \"8c51747e-187c-403e-8bfb-98b13d0d9077\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-64086213-799ffc7fbc-jjd6r"
Oct 30 22:06:11.474177 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:06:11.474141    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8c51747e-187c-403e-8bfb-98b13d0d9077-maas-api-tls\") pod \"maas-api-e2e-ait-64086213-799ffc7fbc-jjd6r\" (UID: \"8c51747e-187c-403e-8bfb-98b13d0d9077\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-64086213-799ffc7fbc-jjd6r"
Oct 30 22:06:11.477005 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:06:11.476983    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8c51747e-187c-403e-8bfb-98b13d0d9077-maas-api-tls\") pod \"maas-api-e2e-ait-64086213-799ffc7fbc-jjd6r\" (UID: \"8c51747e-187c-403e-8bfb-98b13d0d9077\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-64086213-799ffc7fbc-jjd6r"
Oct 30 22:06:11.618646 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:06:11.618606    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-64086213-799ffc7fbc-jjd6r"
Oct 30 22:06:12.002131 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:06:12.002097    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-64086213-799ffc7fbc-jjd6r"]
Oct 30 22:06:12.002572 ip-10-0-140-18 kubenswrapper[4473]: W1030 22:06:12.002326    4473 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c51747e_187c_403e_8bfb_98b13d0d9077.slice/crio-4a9a34fddc8b64a7c4e38f5e810f05c8f9eee31c85f94e3264b4dfb9dd359289 WatchSource:0}: Error finding container 4a9a34fddc8b64a7c4e38f5e810f05c8f9eee31c85f94e3264b4dfb9dd359289: Status 404 returned error can't find the container with id 4a9a34fddc8b64a7c4e38f5e810f05c8f9eee31c85f94e3264b4dfb9dd359289
Oct 30 22:06:12.931693 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:06:12.931656    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-64086213-799ffc7fbc-jjd6r" event={"ID":"8c51747e-187c-403e-8bfb-98b13d0d9077","Type":"ContainerStarted","Data":"4a9a34fddc8b64a7c4e38f5e810f05c8f9eee31c85f94e3264b4dfb9dd359289"}
Oct 30 22:06:13.962075 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:06:13.962002    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-64086213-799ffc7fbc-jjd6r" event={"ID":"8c51747e-187c-403e-8bfb-98b13d0d9077","Type":"ContainerStarted","Data":"308289e6fad1336c9f1504f2c21edb726631162cec05a72b99a794ac4a22c62f"}
Oct 30 22:06:13.964146 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:06:13.964046    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-64086213-799ffc7fbc-jjd6r"
Oct 30 22:06:14.086486 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:06:14.086433    4473 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-64086213-799ffc7fbc-jjd6r" podStartSLOduration=3.134083905 podStartE2EDuration="4.086419879s" totalImagesPullingTime="952.335974ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:06:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:06:14.085165626 +0000 UTC m=+1419.858534113" watchObservedRunningTime="2026-10-30 22:06:14.086419879 +0000 UTC m=+1419.859788366"
Oct 30 22:06:18.987941 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:06:18.980332    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-dde1e8/distinct-2-w0-dde1e8-kserve-744857d57-ftt8b" podUID="577b020d-2a6b-4020-863c-cf19d4f0ca3b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.79:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:06:19.346150 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:06:19.331022    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-0c99b1/distinct-2-w2-0c99b1-kserve-7466864658-8z7fk" podUID="89c09555-cf53-49a7-9c3b-95d2da2a293a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.82:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:06:19.372316 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:06:19.372259    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-dde1e8/distinct-w0-dde1e8-kserve-75b68b4c5d-hts2k" podUID="9d72e7c9-657e-4278-8ed9-53adb75bd530" containerName="main" probeResult="failure" output="Get \"https://10.133.0.72:8000/ready\": context deadline exceeded"
Oct 30 22:06:19.436047 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:06:19.435984    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-0f42cb/distinct-2-w3-0f42cb-kserve-fc477897b-jk8qc" podUID="61b49182-057e-41af-b609-1a6abf82c197" containerName="main" probeResult="failure" output="Get \"https://10.133.0.80:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:06:19.660948 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:06:19.660016    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-29cbad/premium-w6-29cbad-kserve-6c9f8d6899-qtv6c" podUID="381b11f6-3fde-4de7-9ad3-7c0891e76ba1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:06:19.996137 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:06:19.996025    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-dde1e8/distinct-2-w0-dde1e8-kserve-744857d57-ftt8b" podUID="577b020d-2a6b-4020-863c-cf19d4f0ca3b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.79:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:06:20.050954 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:06:20.043142    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-dde1e8/distinct-2-w0-dde1e8-kserve-744857d57-ftt8b"
Oct 30 22:06:20.205997 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:06:20.163034    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-0f42cb/distinct-w3-0f42cb-kserve-fb54fdd59-n6w4l" podUID="ff3c2b25-08aa-4b70-bf57-9cf4a11d05ed" containerName="main" probeResult="failure" output="Get \"https://10.133.0.73:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:06:20.205997 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:06:20.180454    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-9f8aea/distinct-w5-9f8aea-kserve-5b564579cd-kmxzz" podUID="b11e8b11-334f-4a9e-98c2-b0164567a8af" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:06:20.205997 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:06:20.184214    4473 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w6-29cbad/premium-w6-29cbad-kserve-6c9f8d6899-qtv6c" podUID="381b11f6-3fde-4de7-9ad3-7c0891e76ba1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.60:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:06:20.410438 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:06:20.407178    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-dde1e8/distinct-w0-dde1e8-kserve-75b68b4c5d-hts2k" podUID="9d72e7c9-657e-4278-8ed9-53adb75bd530" containerName="main" probeResult="failure" output="Get \"https://10.133.0.72:8000/ready\": context deadline exceeded"
Oct 30 22:06:20.479947 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:06:20.440603    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-0f42cb/distinct-2-w3-0f42cb-kserve-fc477897b-jk8qc" podUID="61b49182-057e-41af-b609-1a6abf82c197" containerName="main" probeResult="failure" output="Get \"https://10.133.0.80:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:06:20.490954 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:06:20.458157    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-0c99b1/distinct-2-w2-0c99b1-kserve-7466864658-8z7fk" podUID="89c09555-cf53-49a7-9c3b-95d2da2a293a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.82:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:06:20.703006 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:06:20.702415    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-29cbad/premium-w6-29cbad-kserve-6c9f8d6899-qtv6c" podUID="381b11f6-3fde-4de7-9ad3-7c0891e76ba1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.60:8000/ready\": context deadline exceeded"
Oct 30 22:06:20.986234 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:06:20.986138    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-dde1e8/distinct-w0-dde1e8-kserve-75b68b4c5d-hts2k"
Oct 30 22:06:20.988941 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:06:20.988900    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-0f42cb/distinct-2-w3-0f42cb-kserve-fc477897b-jk8qc"
Oct 30 22:06:20.989237 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:06:20.989221    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-0c99b1/distinct-2-w2-0c99b1-kserve-7466864658-8z7fk"
Oct 30 22:06:20.989445 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:06:20.989429    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-29cbad/premium-w6-29cbad-kserve-6c9f8d6899-qtv6c"
Oct 30 22:06:21.041167 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:06:21.041103    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-dde1e8/distinct-2-w0-dde1e8-kserve-744857d57-ftt8b" podUID="577b020d-2a6b-4020-863c-cf19d4f0ca3b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.79:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:06:21.101600 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:06:21.101565    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-64086213-799ffc7fbc-jjd6r"
Oct 30 22:06:21.199815 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:06:21.177236    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-0f42cb/distinct-w3-0f42cb-kserve-fb54fdd59-n6w4l" podUID="ff3c2b25-08aa-4b70-bf57-9cf4a11d05ed" containerName="main" probeResult="failure" output="Get \"https://10.133.0.73:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:06:21.201801 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:06:21.201773    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-0f42cb/distinct-w3-0f42cb-kserve-fb54fdd59-n6w4l"
Oct 30 22:06:21.202997 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:06:21.202960    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-9f8aea/distinct-w5-9f8aea-kserve-5b564579cd-kmxzz" podUID="b11e8b11-334f-4a9e-98c2-b0164567a8af" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": context deadline exceeded"
Oct 30 22:06:21.259004 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:06:21.208431    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-9f8aea/distinct-w5-9f8aea-kserve-5b564579cd-kmxzz"
Oct 30 22:06:21.540103 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:06:21.527515    4473 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fncvg" containerName="extractor"
Oct 30 22:06:21.540103 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:06:21.527609    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-dde1e8/distinct-w0-dde1e8-kserve-75b68b4c5d-hts2k" podUID="9d72e7c9-657e-4278-8ed9-53adb75bd530" containerName="main" probeResult="failure" output="Get \"https://10.133.0.72:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:06:21.540103 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:06:21.528231    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-0c99b1/distinct-2-w2-0c99b1-kserve-7466864658-8z7fk" podUID="89c09555-cf53-49a7-9c3b-95d2da2a293a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.82:8000/ready\": context deadline exceeded"
Oct 30 22:06:21.645058 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:06:21.597416    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-0f42cb/distinct-2-w3-0f42cb-kserve-fc477897b-jk8qc" podUID="61b49182-057e-41af-b609-1a6abf82c197" containerName="main" probeResult="failure" output="Get \"https://10.133.0.80:8000/ready\": context deadline exceeded"
Oct 30 22:06:21.672066 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:06:21.645736    4473 patch_prober.go:29] interesting pod/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-j5tqr container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 22:06:21.672066 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:06:21.645797    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-j5tqr" podUID="0f0fbc43-d98b-4b7c-872b-7978f76650ef" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:06:21.724957 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:06:21.719040    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-29cbad/premium-w6-29cbad-kserve-6c9f8d6899-qtv6c" podUID="381b11f6-3fde-4de7-9ad3-7c0891e76ba1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:06:22.083995 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:06:22.079367    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-dde1e8/distinct-2-w0-dde1e8-kserve-744857d57-ftt8b" podUID="577b020d-2a6b-4020-863c-cf19d4f0ca3b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.79:8000/ready\": context deadline exceeded"
Oct 30 22:06:22.133945 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:06:22.118183    4473 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-0c99b1/distinct-2-w2-0c99b1-kserve-7466864658-8z7fk" podUID="89c09555-cf53-49a7-9c3b-95d2da2a293a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.82:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:06:22.230960 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:06:22.177453    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-0f42cb/distinct-w3-0f42cb-kserve-fb54fdd59-n6w4l" podUID="ff3c2b25-08aa-4b70-bf57-9cf4a11d05ed" containerName="main" probeResult="failure" output="Get \"https://10.133.0.73:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:06:22.230960 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:06:22.198944    4473 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-0f42cb/distinct-2-w3-0f42cb-kserve-fc477897b-jk8qc" podUID="61b49182-057e-41af-b609-1a6abf82c197" containerName="main" probeResult="failure" output="Get \"https://10.133.0.80:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:06:22.230960 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:06:22.224242    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-9f8aea/distinct-w5-9f8aea-kserve-5b564579cd-kmxzz" podUID="b11e8b11-334f-4a9e-98c2-b0164567a8af" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": context deadline exceeded"
Oct 30 22:06:22.266947 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:06:22.244973    4473 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-0f42cb/distinct-w3-0f42cb-kserve-fb54fdd59-n6w4l" podUID="ff3c2b25-08aa-4b70-bf57-9cf4a11d05ed" containerName="main" probeResult="failure" output="Get \"https://10.133.0.73:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:06:22.532780 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:06:22.532733    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-dde1e8/distinct-w0-dde1e8-kserve-75b68b4c5d-hts2k" podUID="9d72e7c9-657e-4278-8ed9-53adb75bd530" containerName="main" probeResult="failure" output="Get \"https://10.133.0.72:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:06:22.533455 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:06:22.533419    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-0c99b1/distinct-2-w2-0c99b1-kserve-7466864658-8z7fk" podUID="89c09555-cf53-49a7-9c3b-95d2da2a293a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.82:8000/ready\": context deadline exceeded"
Oct 30 22:06:22.805896 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:06:22.785322    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-0f42cb/distinct-2-w3-0f42cb-kserve-fc477897b-jk8qc" podUID="61b49182-057e-41af-b609-1a6abf82c197" containerName="main" probeResult="failure" output="Get \"https://10.133.0.80:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:06:22.997433 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:06:22.997390    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-29cbad/premium-w6-29cbad-kserve-6c9f8d6899-qtv6c" podUID="381b11f6-3fde-4de7-9ad3-7c0891e76ba1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.60:8000/ready\": context deadline exceeded"
Oct 30 22:06:23.080445 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:06:23.080347    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-dde1e8/distinct-2-w0-dde1e8-kserve-744857d57-ftt8b" podUID="577b020d-2a6b-4020-863c-cf19d4f0ca3b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.79:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:06:23.192966 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:06:23.180621    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-0f42cb/distinct-w3-0f42cb-kserve-fb54fdd59-n6w4l" podUID="ff3c2b25-08aa-4b70-bf57-9cf4a11d05ed" containerName="main" probeResult="failure" output="Get \"https://10.133.0.73:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:06:23.229003 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:06:23.228959    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-9f8aea/distinct-w5-9f8aea-kserve-5b564579cd-kmxzz" podUID="b11e8b11-334f-4a9e-98c2-b0164567a8af" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:06:23.599950 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:06:23.599774    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-dde1e8/distinct-w0-dde1e8-kserve-75b68b4c5d-hts2k" podUID="9d72e7c9-657e-4278-8ed9-53adb75bd530" containerName="main" probeResult="failure" output="Get \"https://10.133.0.72:8000/ready\": context deadline exceeded"
Oct 30 22:06:23.647883 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:06:23.647853    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-0f42cb/distinct-2-w3-0f42cb-kserve-fc477897b-jk8qc"
Oct 30 22:06:23.667088 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:06:23.649033    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-0c99b1/distinct-2-w2-0c99b1-kserve-7466864658-8z7fk" podUID="89c09555-cf53-49a7-9c3b-95d2da2a293a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.82:8000/ready\": context deadline exceeded"
Oct 30 22:06:23.735132 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:06:23.735026    4473 patch_prober.go:29] interesting pod/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-l8zrp container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.90:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 22:06:23.735613 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:06:23.735358    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-l8zrp" podUID="78f06b05-32bf-46bc-bd95-2b3a8da72b5b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.90:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:06:23.960953 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:06:23.943587    4473 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-dde1e8/distinct-w0-dde1e8-kserve-75b68b4c5d-hts2k" podUID="9d72e7c9-657e-4278-8ed9-53adb75bd530" containerName="main" probeResult="failure" output="Get \"https://10.133.0.72:8000/health\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:06:24.029361 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:06:24.018237    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-29cbad/premium-w6-29cbad-kserve-6c9f8d6899-qtv6c" podUID="381b11f6-3fde-4de7-9ad3-7c0891e76ba1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.60:8000/ready\": context deadline exceeded"
Oct 30 22:06:24.088942 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:06:24.085048    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-dde1e8/distinct-2-w0-dde1e8-kserve-744857d57-ftt8b" podUID="577b020d-2a6b-4020-863c-cf19d4f0ca3b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.79:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:06:24.142435 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:06:24.139470    4473 patch_prober.go:29] interesting pod/e2e-worker-w2-0c99b1-openshift-default-967855977-ktzwz container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.86:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 22:06:24.142435 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:06:24.139553    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-ktzwz" podUID="df9bbdb1-5a45-4223-b03a-5d5e11542e60" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.86:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:06:24.276028 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:06:24.268496    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-29cbad/premium-w6-29cbad-kserve-6c9f8d6899-qtv6c"
Oct 30 22:06:24.307521 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:06:24.304042    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-9f8aea/distinct-w5-9f8aea-kserve-5b564579cd-kmxzz" podUID="b11e8b11-334f-4a9e-98c2-b0164567a8af" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:06:24.520072 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:06:24.520028    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-0f42cb/distinct-w3-0f42cb-kserve-fb54fdd59-n6w4l" podUID="ff3c2b25-08aa-4b70-bf57-9cf4a11d05ed" containerName="main" probeResult="failure" output="Get \"https://10.133.0.73:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:06:24.630302 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:06:24.630259    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-dde1e8/distinct-w0-dde1e8-kserve-75b68b4c5d-hts2k" podUID="9d72e7c9-657e-4278-8ed9-53adb75bd530" containerName="main" probeResult="failure" output="Get \"https://10.133.0.72:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:06:24.689944 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:06:24.662130    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-0f42cb/distinct-2-w3-0f42cb-kserve-fc477897b-jk8qc" podUID="61b49182-057e-41af-b609-1a6abf82c197" containerName="main" probeResult="failure" output="Get \"https://10.133.0.80:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:06:24.992005 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:06:24.923576    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-dde1e8/distinct-w0-dde1e8-kserve-75b68b4c5d-hts2k"
Oct 30 22:06:25.114236 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:06:25.114181    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-0c99b1/distinct-2-w2-0c99b1-kserve-7466864658-8z7fk" podUID="89c09555-cf53-49a7-9c3b-95d2da2a293a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.82:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:06:25.175652 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:06:25.138472    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-dde1e8/distinct-2-w0-dde1e8-kserve-744857d57-ftt8b" podUID="577b020d-2a6b-4020-863c-cf19d4f0ca3b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.79:8000/ready\": context deadline exceeded"
Oct 30 22:06:25.237065 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:06:25.233544    4473 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-dde1e8/distinct-2-w0-dde1e8-kserve-744857d57-ftt8b" podUID="577b020d-2a6b-4020-863c-cf19d4f0ca3b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.79:8000/health\": context deadline exceeded"
Oct 30 22:06:25.323187 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:06:25.323083    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-9f8aea/distinct-w5-9f8aea-kserve-5b564579cd-kmxzz" podUID="b11e8b11-334f-4a9e-98c2-b0164567a8af" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": context deadline exceeded"
Oct 30 22:06:25.527277 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:06:25.527227    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-0f42cb/distinct-w3-0f42cb-kserve-fb54fdd59-n6w4l" podUID="ff3c2b25-08aa-4b70-bf57-9cf4a11d05ed" containerName="main" probeResult="failure" output="Get \"https://10.133.0.73:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:06:25.752948 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:06:25.686322    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-0f42cb/distinct-2-w3-0f42cb-kserve-fc477897b-jk8qc" podUID="61b49182-057e-41af-b609-1a6abf82c197" containerName="main" probeResult="failure" output="Get \"https://10.133.0.80:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:06:25.752948 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:06:25.749645    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-0f42cb/distinct-2-w3-0f42cb-kserve-fc477897b-jk8qc"
Oct 30 22:06:25.810581 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:06:25.793061    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-29cbad/premium-w6-29cbad-kserve-6c9f8d6899-qtv6c" podUID="381b11f6-3fde-4de7-9ad3-7c0891e76ba1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.60:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:06:26.168290 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:06:26.166111    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-dde1e8/distinct-2-w0-dde1e8-kserve-744857d57-ftt8b" podUID="577b020d-2a6b-4020-863c-cf19d4f0ca3b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.79:8000/ready\": context deadline exceeded"
Oct 30 22:06:26.239366 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:06:26.239295    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-0c99b1/distinct-2-w2-0c99b1-kserve-7466864658-8z7fk" podUID="89c09555-cf53-49a7-9c3b-95d2da2a293a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.82:8000/ready\": context deadline exceeded"
Oct 30 22:06:26.303768 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:06:26.303741    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-0f42cb/distinct-2-w3-0f42cb-kserve-fc477897b-jk8qc"
Oct 30 22:06:26.334243 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:06:26.334178    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-9f8aea/distinct-w5-9f8aea-kserve-5b564579cd-kmxzz" podUID="b11e8b11-334f-4a9e-98c2-b0164567a8af" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:06:26.680008 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:06:26.598459    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-9f8aea/distinct-w5-9f8aea-kserve-5b564579cd-kmxzz"
Oct 30 22:06:26.870975 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:06:26.868795    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-0f42cb/distinct-w3-0f42cb-kserve-fb54fdd59-n6w4l"
Oct 30 22:06:26.912616 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:06:26.912593    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-dde1e8/distinct-2-w0-dde1e8-kserve-744857d57-ftt8b"
Oct 30 22:06:26.956420 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:06:26.956332    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-0c99b1/distinct-2-w2-0c99b1-kserve-7466864658-8z7fk"
Oct 30 22:06:34.130836 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:06:34.130802    4473 generic.go:356] "Generic (PLEG): container finished" podID="071132ee-06b1-4b98-9905-6afd31e4289b" containerID="43a88e25ce00373dfe7cf2bb371405728c2598ac4dfd8589bb08daaf117651ea" exitCode=0
Oct 30 22:06:34.131412 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:06:34.130881    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-1d71f464-openshift-default-55f9d6d6bc-c4fqx" event={"ID":"071132ee-06b1-4b98-9905-6afd31e4289b","Type":"ContainerDied","Data":"43a88e25ce00373dfe7cf2bb371405728c2598ac4dfd8589bb08daaf117651ea"}
Oct 30 22:06:34.131412 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:06:34.131379    4473 scope.go:122] "RemoveContainer" containerID="43a88e25ce00373dfe7cf2bb371405728c2598ac4dfd8589bb08daaf117651ea"
Oct 30 22:06:34.133105 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:06:34.133072    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:06:34.133228 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:06:34.133145    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:06:34.133228 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:06:34.133180    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:06:35.155944 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:06:35.152274    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-1d71f464-openshift-default-55f9d6d6bc-c4fqx" event={"ID":"071132ee-06b1-4b98-9905-6afd31e4289b","Type":"ContainerStarted","Data":"5031517fce58bf8bdcb380d35c8c386e94aac5fb6392c8ef1ca19d79233e723f"}
Oct 30 22:06:36.062790 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:06:36.062760    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-1d71f464-openshift-default-55f9d6d6bc-c4fqx"
Oct 30 22:06:36.063061 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:06:36.063021    4473 patch_prober.go:29] interesting pod/e2e-ait-1d71f464-openshift-default-55f9d6d6bc-c4fqx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.91:15021/healthz/ready\": dial tcp 10.133.0.91:15021: connect: connection refused" start-of-body=
Oct 30 22:06:36.063202 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:06:36.063075    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-1d71f464-openshift-default-55f9d6d6bc-c4fqx" podUID="071132ee-06b1-4b98-9905-6afd31e4289b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.91:15021/healthz/ready\": dial tcp 10.133.0.91:15021: connect: connection refused"
Oct 30 22:06:37.063248 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:06:37.063215    4473 patch_prober.go:29] interesting pod/e2e-ait-1d71f464-openshift-default-55f9d6d6bc-c4fqx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.91:15021/healthz/ready\": dial tcp 10.133.0.91:15021: connect: connection refused" start-of-body=
Oct 30 22:06:37.063666 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:06:37.063276    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-1d71f464-openshift-default-55f9d6d6bc-c4fqx" podUID="071132ee-06b1-4b98-9905-6afd31e4289b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.91:15021/healthz/ready\": dial tcp 10.133.0.91:15021: connect: connection refused"
Oct 30 22:06:38.063253 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:06:38.063219    4473 patch_prober.go:29] interesting pod/e2e-ait-1d71f464-openshift-default-55f9d6d6bc-c4fqx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.91:15021/healthz/ready\": dial tcp 10.133.0.91:15021: connect: connection refused" start-of-body=
Oct 30 22:06:38.063656 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:06:38.063278    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-1d71f464-openshift-default-55f9d6d6bc-c4fqx" podUID="071132ee-06b1-4b98-9905-6afd31e4289b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.91:15021/healthz/ready\": dial tcp 10.133.0.91:15021: connect: connection refused"
Oct 30 22:06:39.063665 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:06:39.063633    4473 patch_prober.go:29] interesting pod/e2e-ait-1d71f464-openshift-default-55f9d6d6bc-c4fqx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.91:15021/healthz/ready\": dial tcp 10.133.0.91:15021: connect: connection refused" start-of-body=
Oct 30 22:06:39.064110 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:06:39.063698    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-1d71f464-openshift-default-55f9d6d6bc-c4fqx" podUID="071132ee-06b1-4b98-9905-6afd31e4289b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.91:15021/healthz/ready\": dial tcp 10.133.0.91:15021: connect: connection refused"
Oct 30 22:06:40.063149 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:06:40.063117    4473 patch_prober.go:29] interesting pod/e2e-ait-1d71f464-openshift-default-55f9d6d6bc-c4fqx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.91:15021/healthz/ready\": dial tcp 10.133.0.91:15021: connect: connection refused" start-of-body=
Oct 30 22:06:40.063335 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:06:40.063173    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-1d71f464-openshift-default-55f9d6d6bc-c4fqx" podUID="071132ee-06b1-4b98-9905-6afd31e4289b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.91:15021/healthz/ready\": dial tcp 10.133.0.91:15021: connect: connection refused"
Oct 30 22:06:41.068004 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:06:41.067971    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-1d71f464-openshift-default-55f9d6d6bc-c4fqx"
Oct 30 22:06:41.068417 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:06:41.068397    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-1d71f464-openshift-default-55f9d6d6bc-c4fqx"
Oct 30 22:06:41.069283 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:06:41.069264    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-1d71f464-openshift-default-55f9d6d6bc-c4fqx"
Oct 30 22:06:50.327264 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:06:50.327222    4473 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fncvg" containerName="extractor"
Oct 30 22:07:13.556567 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:07:13.556532    4473 generic.go:356] "Generic (PLEG): container finished" podID="8e2ea507-72cc-48c6-87d8-6f1805fbfd43" containerID="1f22a165891cfc9cc1f2a87cb106cd84e60b3f6cd65674c726c8eada468802bf" exitCode=0
Oct 30 22:07:13.557163 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:07:13.556608    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-64086213-openshift-default-7575697f5c-dszmm" event={"ID":"8e2ea507-72cc-48c6-87d8-6f1805fbfd43","Type":"ContainerDied","Data":"1f22a165891cfc9cc1f2a87cb106cd84e60b3f6cd65674c726c8eada468802bf"}
Oct 30 22:07:13.557273 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:07:13.557251    4473 scope.go:122] "RemoveContainer" containerID="1f22a165891cfc9cc1f2a87cb106cd84e60b3f6cd65674c726c8eada468802bf"
Oct 30 22:07:13.558436 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:07:13.558415    4473 generic.go:356] "Generic (PLEG): container finished" podID="e85a3a56-ad99-4c7f-bfbf-c858e78f1dd0" containerID="56b99d959e478fff249b8fa8302ee589f3144a062ebd32d58e9aa8e87605f699" exitCode=0
Oct 30 22:07:13.558552 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:07:13.558481    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-97895160-openshift-default-5f45687f99-pzfmz" event={"ID":"e85a3a56-ad99-4c7f-bfbf-c858e78f1dd0","Type":"ContainerDied","Data":"56b99d959e478fff249b8fa8302ee589f3144a062ebd32d58e9aa8e87605f699"}
Oct 30 22:07:13.558869 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:07:13.558855    4473 scope.go:122] "RemoveContainer" containerID="56b99d959e478fff249b8fa8302ee589f3144a062ebd32d58e9aa8e87605f699"
Oct 30 22:07:13.559097 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:07:13.559070    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:07:13.559154 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:07:13.559140    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:07:13.559195 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:07:13.559173    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:07:13.560452 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:07:13.560422    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:07:13.560547 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:07:13.560488    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:07:13.560547 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:07:13.560524    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:07:14.633137 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:07:14.633092    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-64086213-openshift-default-7575697f5c-dszmm" event={"ID":"8e2ea507-72cc-48c6-87d8-6f1805fbfd43","Type":"ContainerStarted","Data":"2f93aff36996f899f826a5c00c05d6a58d7af87fa25802b7dcbc4ac3ac9a97b3"}
Oct 30 22:07:14.657857 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:07:14.657819    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-97895160-openshift-default-5f45687f99-pzfmz" event={"ID":"e85a3a56-ad99-4c7f-bfbf-c858e78f1dd0","Type":"ContainerStarted","Data":"c7b5053c3decda8b882ea24f9c7646d6237816c165e5008dc67da0e23bccf3b4"}
Oct 30 22:07:15.224250 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:07:15.224213    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-97895160-openshift-default-5f45687f99-pzfmz"
Oct 30 22:07:15.224446 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:07:15.224396    4473 patch_prober.go:29] interesting pod/e2e-ait-97895160-openshift-default-5f45687f99-pzfmz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.94:15021/healthz/ready\": dial tcp 10.133.0.94:15021: connect: connection refused" start-of-body=
Oct 30 22:07:15.224492 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:07:15.224468    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-97895160-openshift-default-5f45687f99-pzfmz" podUID="e85a3a56-ad99-4c7f-bfbf-c858e78f1dd0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.94:15021/healthz/ready\": dial tcp 10.133.0.94:15021: connect: connection refused"
Oct 30 22:07:15.448946 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:07:15.443112    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-64086213-openshift-default-7575697f5c-dszmm"
Oct 30 22:07:15.448946 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:07:15.445259    4473 patch_prober.go:29] interesting pod/e2e-ait-64086213-openshift-default-7575697f5c-dszmm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.92:15021/healthz/ready\": dial tcp 10.133.0.92:15021: connect: connection refused" start-of-body=
Oct 30 22:07:15.448946 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:07:15.445308    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-64086213-openshift-default-7575697f5c-dszmm" podUID="8e2ea507-72cc-48c6-87d8-6f1805fbfd43" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.92:15021/healthz/ready\": dial tcp 10.133.0.92:15021: connect: connection refused"
Oct 30 22:07:16.224261 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:07:16.224227    4473 patch_prober.go:29] interesting pod/e2e-ait-97895160-openshift-default-5f45687f99-pzfmz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.94:15021/healthz/ready\": dial tcp 10.133.0.94:15021: connect: connection refused" start-of-body=
Oct 30 22:07:16.224645 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:07:16.224294    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-97895160-openshift-default-5f45687f99-pzfmz" podUID="e85a3a56-ad99-4c7f-bfbf-c858e78f1dd0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.94:15021/healthz/ready\": dial tcp 10.133.0.94:15021: connect: connection refused"
Oct 30 22:07:16.453947 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:07:16.450138    4473 patch_prober.go:29] interesting pod/e2e-ait-64086213-openshift-default-7575697f5c-dszmm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.92:15021/healthz/ready\": dial tcp 10.133.0.92:15021: connect: connection refused" start-of-body=
Oct 30 22:07:16.453947 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:07:16.450213    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-64086213-openshift-default-7575697f5c-dszmm" podUID="8e2ea507-72cc-48c6-87d8-6f1805fbfd43" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.92:15021/healthz/ready\": dial tcp 10.133.0.92:15021: connect: connection refused"
Oct 30 22:07:17.224624 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:07:17.224348    4473 patch_prober.go:29] interesting pod/e2e-ait-97895160-openshift-default-5f45687f99-pzfmz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.94:15021/healthz/ready\": dial tcp 10.133.0.94:15021: connect: connection refused" start-of-body=
Oct 30 22:07:17.224624 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:07:17.224403    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-97895160-openshift-default-5f45687f99-pzfmz" podUID="e85a3a56-ad99-4c7f-bfbf-c858e78f1dd0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.94:15021/healthz/ready\": dial tcp 10.133.0.94:15021: connect: connection refused"
Oct 30 22:07:17.455947 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:07:17.450379    4473 patch_prober.go:29] interesting pod/e2e-ait-64086213-openshift-default-7575697f5c-dszmm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.92:15021/healthz/ready\": dial tcp 10.133.0.92:15021: connect: connection refused" start-of-body=
Oct 30 22:07:17.455947 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:07:17.450453    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-64086213-openshift-default-7575697f5c-dszmm" podUID="8e2ea507-72cc-48c6-87d8-6f1805fbfd43" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.92:15021/healthz/ready\": dial tcp 10.133.0.92:15021: connect: connection refused"
Oct 30 22:07:18.050064 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:07:18.050027    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-64086213-openshift-default-7575697f5c-dszmm"
Oct 30 22:07:18.224280 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:07:18.224250    4473 patch_prober.go:29] interesting pod/e2e-ait-97895160-openshift-default-5f45687f99-pzfmz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.94:15021/healthz/ready\": dial tcp 10.133.0.94:15021: connect: connection refused" start-of-body=
Oct 30 22:07:18.224461 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:07:18.224318    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-97895160-openshift-default-5f45687f99-pzfmz" podUID="e85a3a56-ad99-4c7f-bfbf-c858e78f1dd0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.94:15021/healthz/ready\": dial tcp 10.133.0.94:15021: connect: connection refused"
Oct 30 22:07:18.285745 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:07:18.285709    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-1d71f464-5df797dcdc-wwkgc"]
Oct 30 22:07:18.443389 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:07:18.443348    4473 patch_prober.go:29] interesting pod/e2e-ait-64086213-openshift-default-7575697f5c-dszmm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.92:15021/healthz/ready\": dial tcp 10.133.0.92:15021: connect: connection refused" start-of-body=
Oct 30 22:07:18.443686 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:07:18.443656    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-64086213-openshift-default-7575697f5c-dszmm" podUID="8e2ea507-72cc-48c6-87d8-6f1805fbfd43" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.92:15021/healthz/ready\": dial tcp 10.133.0.92:15021: connect: connection refused"
Oct 30 22:07:19.224005 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:07:19.223970    4473 patch_prober.go:29] interesting pod/e2e-ait-97895160-openshift-default-5f45687f99-pzfmz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.94:15021/healthz/ready\": dial tcp 10.133.0.94:15021: connect: connection refused" start-of-body=
Oct 30 22:07:19.224227 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:07:19.224035    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-97895160-openshift-default-5f45687f99-pzfmz" podUID="e85a3a56-ad99-4c7f-bfbf-c858e78f1dd0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.94:15021/healthz/ready\": dial tcp 10.133.0.94:15021: connect: connection refused"
Oct 30 22:07:19.457422 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:07:19.457380    4473 patch_prober.go:29] interesting pod/e2e-ait-64086213-openshift-default-7575697f5c-dszmm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.92:15021/healthz/ready\": dial tcp 10.133.0.92:15021: connect: connection refused" start-of-body=
Oct 30 22:07:19.458087 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:07:19.458050    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-64086213-openshift-default-7575697f5c-dszmm" podUID="8e2ea507-72cc-48c6-87d8-6f1805fbfd43" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.92:15021/healthz/ready\": dial tcp 10.133.0.92:15021: connect: connection refused"
Oct 30 22:07:20.224914 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:07:20.224878    4473 patch_prober.go:29] interesting pod/e2e-ait-97895160-openshift-default-5f45687f99-pzfmz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.94:15021/healthz/ready\": dial tcp 10.133.0.94:15021: connect: connection refused" start-of-body=
Oct 30 22:07:20.225246 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:07:20.225196    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-97895160-openshift-default-5f45687f99-pzfmz" podUID="e85a3a56-ad99-4c7f-bfbf-c858e78f1dd0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.94:15021/healthz/ready\": dial tcp 10.133.0.94:15021: connect: connection refused"
Oct 30 22:07:20.486105 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:07:20.486016    4473 patch_prober.go:29] interesting pod/e2e-ait-64086213-openshift-default-7575697f5c-dszmm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.92:15021/healthz/ready\": dial tcp 10.133.0.92:15021: connect: connection refused" start-of-body=
Oct 30 22:07:20.486669 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:07:20.486635    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-64086213-openshift-default-7575697f5c-dszmm" podUID="8e2ea507-72cc-48c6-87d8-6f1805fbfd43" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.92:15021/healthz/ready\": dial tcp 10.133.0.92:15021: connect: connection refused"
Oct 30 22:07:20.500377 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:07:20.500347    4473 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fncvg" containerName="extractor"
Oct 30 22:07:21.229263 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:07:21.229228    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-97895160-openshift-default-5f45687f99-pzfmz"
Oct 30 22:07:21.229762 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:07:21.229736    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-97895160-openshift-default-5f45687f99-pzfmz"
Oct 30 22:07:21.230805 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:07:21.230785    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-97895160-openshift-default-5f45687f99-pzfmz"
Oct 30 22:07:21.446935 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:07:21.446884    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-64086213-openshift-default-7575697f5c-dszmm"
Oct 30 22:07:21.448105 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:07:21.448084    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-64086213-openshift-default-7575697f5c-dszmm"
Oct 30 22:07:23.286817 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:07:23.286767    4473 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-1d71f464-5df797dcdc-wwkgc" podUID="6f549a4c-acb7-46b6-85bb-65a7dd17a503" containerName="payload-pre-processing" containerID="cri-o://c7830359eab1970672d4dc015db0518c7ed878f0a816d5ab9a7abffad2980ad3" gracePeriod=25
Oct 30 22:07:23.611371 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:07:23.611341    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-1d71f464-5df797dcdc-wwkgc"
Oct 30 22:07:23.667119 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:07:23.667088    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6f549a4c-acb7-46b6-85bb-65a7dd17a503-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6f549a4c-acb7-46b6-85bb-65a7dd17a503-plugins-config-volume\") pod \"6f549a4c-acb7-46b6-85bb-65a7dd17a503\" (UID: \"6f549a4c-acb7-46b6-85bb-65a7dd17a503\") "
Oct 30 22:07:23.667338 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:07:23.667267    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6f549a4c-acb7-46b6-85bb-65a7dd17a503-kube-api-access-4x7sn\" (UniqueName: \"kubernetes.io/projected/6f549a4c-acb7-46b6-85bb-65a7dd17a503-kube-api-access-4x7sn\") pod \"6f549a4c-acb7-46b6-85bb-65a7dd17a503\" (UID: \"6f549a4c-acb7-46b6-85bb-65a7dd17a503\") "
Oct 30 22:07:23.667551 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:07:23.667524    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6f549a4c-acb7-46b6-85bb-65a7dd17a503-plugins-config-volume" pod "6f549a4c-acb7-46b6-85bb-65a7dd17a503" (UID: "6f549a4c-acb7-46b6-85bb-65a7dd17a503"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:07:23.667984 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:07:23.667963    4473 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6f549a4c-acb7-46b6-85bb-65a7dd17a503-plugins-config-volume\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:07:23.669725 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:07:23.669693    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6f549a4c-acb7-46b6-85bb-65a7dd17a503-kube-api-access-4x7sn" pod "6f549a4c-acb7-46b6-85bb-65a7dd17a503" (UID: "6f549a4c-acb7-46b6-85bb-65a7dd17a503"). InnerVolumeSpecName "kube-api-access-4x7sn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:07:23.769589 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:07:23.769544    4473 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4x7sn\" (UniqueName: \"kubernetes.io/projected/6f549a4c-acb7-46b6-85bb-65a7dd17a503-kube-api-access-4x7sn\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:07:24.039937 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:07:24.039885    4473 generic.go:356] "Generic (PLEG): container finished" podID="6f549a4c-acb7-46b6-85bb-65a7dd17a503" containerID="c7830359eab1970672d4dc015db0518c7ed878f0a816d5ab9a7abffad2980ad3" exitCode=0
Oct 30 22:07:24.040134 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:07:24.039982    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-1d71f464-5df797dcdc-wwkgc"
Oct 30 22:07:24.040134 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:07:24.039980    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-1d71f464-5df797dcdc-wwkgc" event={"ID":"6f549a4c-acb7-46b6-85bb-65a7dd17a503","Type":"ContainerDied","Data":"c7830359eab1970672d4dc015db0518c7ed878f0a816d5ab9a7abffad2980ad3"}
Oct 30 22:07:24.040134 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:07:24.040119    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-1d71f464-5df797dcdc-wwkgc" event={"ID":"6f549a4c-acb7-46b6-85bb-65a7dd17a503","Type":"ContainerDied","Data":"8a241753c70e17c9933b083ddfd9bbb8e1878cff86fa45a0229bd4fdf05e16fb"}
Oct 30 22:07:24.040309 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:07:24.040151    4473 scope.go:122] "RemoveContainer" containerID="c7830359eab1970672d4dc015db0518c7ed878f0a816d5ab9a7abffad2980ad3"
Oct 30 22:07:24.054884 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:07:24.054856    4473 scope.go:122] "RemoveContainer" containerID="c7830359eab1970672d4dc015db0518c7ed878f0a816d5ab9a7abffad2980ad3"
Oct 30 22:07:24.055273 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:07:24.055244    4473 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c7830359eab1970672d4dc015db0518c7ed878f0a816d5ab9a7abffad2980ad3\": container with ID starting with c7830359eab1970672d4dc015db0518c7ed878f0a816d5ab9a7abffad2980ad3 not found: ID does not exist" containerID="c7830359eab1970672d4dc015db0518c7ed878f0a816d5ab9a7abffad2980ad3"
Oct 30 22:07:24.055356 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:07:24.055285    4473 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c7830359eab1970672d4dc015db0518c7ed878f0a816d5ab9a7abffad2980ad3"} err="failed to get container status \"c7830359eab1970672d4dc015db0518c7ed878f0a816d5ab9a7abffad2980ad3\": rpc error: code = NotFound desc = could not find container \"c7830359eab1970672d4dc015db0518c7ed878f0a816d5ab9a7abffad2980ad3\": container with ID starting with c7830359eab1970672d4dc015db0518c7ed878f0a816d5ab9a7abffad2980ad3 not found: ID does not exist"
Oct 30 22:07:24.207587 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:07:24.207552    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-1d71f464-5df797dcdc-wwkgc"]
Oct 30 22:07:24.258623 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:07:24.258574    4473 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-1d71f464-5df797dcdc-wwkgc"]
Oct 30 22:07:24.359631 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:07:24.359546    4473 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6f549a4c-acb7-46b6-85bb-65a7dd17a503" path="/var/lib/kubelet/pods/6f549a4c-acb7-46b6-85bb-65a7dd17a503/volumes"
Oct 30 22:07:44.056618 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:07:44.056582    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-97895160-784c6ff67f-6xks4"]
Oct 30 22:07:49.057220 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:07:49.057162    4473 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-97895160-784c6ff67f-6xks4" podUID="fe4f0d92-1deb-445e-b700-2e62abd9e4e3" containerName="payload-pre-processing" containerID="cri-o://7e5a30b9de1aa8e18968961e36c18bbc24da831ca1465aa4e3b69b1f4e3ab3ba" gracePeriod=25
Oct 30 22:07:49.252853 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:07:49.252817    4473 generic.go:356] "Generic (PLEG): container finished" podID="fe4f0d92-1deb-445e-b700-2e62abd9e4e3" containerID="7e5a30b9de1aa8e18968961e36c18bbc24da831ca1465aa4e3b69b1f4e3ab3ba" exitCode=0
Oct 30 22:07:49.253090 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:07:49.253065    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-97895160-784c6ff67f-6xks4" event={"ID":"fe4f0d92-1deb-445e-b700-2e62abd9e4e3","Type":"ContainerDied","Data":"7e5a30b9de1aa8e18968961e36c18bbc24da831ca1465aa4e3b69b1f4e3ab3ba"}
Oct 30 22:07:49.356301 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:07:49.356273    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-97895160-784c6ff67f-6xks4"
Oct 30 22:07:49.455385 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:07:49.455342    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fe4f0d92-1deb-445e-b700-2e62abd9e4e3-kube-api-access-ql6hs\" (UniqueName: \"kubernetes.io/projected/fe4f0d92-1deb-445e-b700-2e62abd9e4e3-kube-api-access-ql6hs\") pod \"fe4f0d92-1deb-445e-b700-2e62abd9e4e3\" (UID: \"fe4f0d92-1deb-445e-b700-2e62abd9e4e3\") "
Oct 30 22:07:49.455596 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:07:49.455580    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fe4f0d92-1deb-445e-b700-2e62abd9e4e3-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fe4f0d92-1deb-445e-b700-2e62abd9e4e3-plugins-config-volume\") pod \"fe4f0d92-1deb-445e-b700-2e62abd9e4e3\" (UID: \"fe4f0d92-1deb-445e-b700-2e62abd9e4e3\") "
Oct 30 22:07:49.456057 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:07:49.456024    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fe4f0d92-1deb-445e-b700-2e62abd9e4e3-plugins-config-volume" pod "fe4f0d92-1deb-445e-b700-2e62abd9e4e3" (UID: "fe4f0d92-1deb-445e-b700-2e62abd9e4e3"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:07:49.456746 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:07:49.456727    4473 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fe4f0d92-1deb-445e-b700-2e62abd9e4e3-plugins-config-volume\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:07:49.458032 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:07:49.458006    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fe4f0d92-1deb-445e-b700-2e62abd9e4e3-kube-api-access-ql6hs" pod "fe4f0d92-1deb-445e-b700-2e62abd9e4e3" (UID: "fe4f0d92-1deb-445e-b700-2e62abd9e4e3"). InnerVolumeSpecName "kube-api-access-ql6hs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:07:49.558498 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:07:49.558464    4473 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ql6hs\" (UniqueName: \"kubernetes.io/projected/fe4f0d92-1deb-445e-b700-2e62abd9e4e3-kube-api-access-ql6hs\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:07:50.261910 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:07:50.261879    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-97895160-784c6ff67f-6xks4" event={"ID":"fe4f0d92-1deb-445e-b700-2e62abd9e4e3","Type":"ContainerDied","Data":"b361fcc52cb42837e87768ade8e3bb08deec5c459380cf41505e6673ffefcc3b"}
Oct 30 22:07:50.262377 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:07:50.261912    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-97895160-784c6ff67f-6xks4"
Oct 30 22:07:50.262377 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:07:50.261996    4473 scope.go:122] "RemoveContainer" containerID="7e5a30b9de1aa8e18968961e36c18bbc24da831ca1465aa4e3b69b1f4e3ab3ba"
Oct 30 22:07:50.328518 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:07:50.328470    4473 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fncvg" containerName="extractor"
Oct 30 22:07:50.430613 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:07:50.430581    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-97895160-784c6ff67f-6xks4"]
Oct 30 22:07:50.480041 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:07:50.480006    4473 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-97895160-784c6ff67f-6xks4"]
Oct 30 22:07:52.359790 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:07:52.359751    4473 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fe4f0d92-1deb-445e-b700-2e62abd9e4e3" path="/var/lib/kubelet/pods/fe4f0d92-1deb-445e-b700-2e62abd9e4e3/volumes"
Oct 30 22:07:57.636445 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:07:57.636401    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-1d71f464-7b656bc7b4-pqlb6"]
Oct 30 22:07:57.636882 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:07:57.636630    4473 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1d71f464-7b656bc7b4-pqlb6" podUID="976cc05a-058b-43d0-95f6-75e6f0425e2f" containerName="maas-api" containerID="cri-o://d6b59061e22da417c7cf5b73e528603ce9e69c902534caedb8187a6fbe0ffc6d" gracePeriod=30
Oct 30 22:07:58.363685 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:07:58.363439    4473 generic.go:356] "Generic (PLEG): container finished" podID="976cc05a-058b-43d0-95f6-75e6f0425e2f" containerID="d6b59061e22da417c7cf5b73e528603ce9e69c902534caedb8187a6fbe0ffc6d" exitCode=0
Oct 30 22:07:58.367515 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:07:58.367384    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1d71f464-7b656bc7b4-pqlb6"
Oct 30 22:07:58.408944 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:07:58.404336    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1d71f464-7b656bc7b4-pqlb6" event={"ID":"976cc05a-058b-43d0-95f6-75e6f0425e2f","Type":"ContainerDied","Data":"d6b59061e22da417c7cf5b73e528603ce9e69c902534caedb8187a6fbe0ffc6d"}
Oct 30 22:07:58.408944 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:07:58.404412    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1d71f464-7b656bc7b4-pqlb6" event={"ID":"976cc05a-058b-43d0-95f6-75e6f0425e2f","Type":"ContainerDied","Data":"402401d3378953e78fc358ba38caaebea54f532d07a0a04a925b972f3781792b"}
Oct 30 22:07:58.408944 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:07:58.404449    4473 scope.go:122] "RemoveContainer" containerID="d6b59061e22da417c7cf5b73e528603ce9e69c902534caedb8187a6fbe0ffc6d"
Oct 30 22:07:58.427237 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:07:58.426859    4473 scope.go:122] "RemoveContainer" containerID="d6b59061e22da417c7cf5b73e528603ce9e69c902534caedb8187a6fbe0ffc6d"
Oct 30 22:07:58.427634 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:07:58.427524    4473 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d6b59061e22da417c7cf5b73e528603ce9e69c902534caedb8187a6fbe0ffc6d\": container with ID starting with d6b59061e22da417c7cf5b73e528603ce9e69c902534caedb8187a6fbe0ffc6d not found: ID does not exist" containerID="d6b59061e22da417c7cf5b73e528603ce9e69c902534caedb8187a6fbe0ffc6d"
Oct 30 22:07:58.427634 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:07:58.427567    4473 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d6b59061e22da417c7cf5b73e528603ce9e69c902534caedb8187a6fbe0ffc6d"} err="failed to get container status \"d6b59061e22da417c7cf5b73e528603ce9e69c902534caedb8187a6fbe0ffc6d\": rpc error: code = NotFound desc = could not find container \"d6b59061e22da417c7cf5b73e528603ce9e69c902534caedb8187a6fbe0ffc6d\": container with ID starting with d6b59061e22da417c7cf5b73e528603ce9e69c902534caedb8187a6fbe0ffc6d not found: ID does not exist"
Oct 30 22:07:58.489216 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:07:58.489176    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/976cc05a-058b-43d0-95f6-75e6f0425e2f-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/976cc05a-058b-43d0-95f6-75e6f0425e2f-openshift-service-ca\") pod \"976cc05a-058b-43d0-95f6-75e6f0425e2f\" (UID: \"976cc05a-058b-43d0-95f6-75e6f0425e2f\") "
Oct 30 22:07:58.489469 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:07:58.489446    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/976cc05a-058b-43d0-95f6-75e6f0425e2f-kube-api-access-n7bdw\" (UniqueName: \"kubernetes.io/projected/976cc05a-058b-43d0-95f6-75e6f0425e2f-kube-api-access-n7bdw\") pod \"976cc05a-058b-43d0-95f6-75e6f0425e2f\" (UID: \"976cc05a-058b-43d0-95f6-75e6f0425e2f\") "
Oct 30 22:07:58.489539 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:07:58.489490    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/976cc05a-058b-43d0-95f6-75e6f0425e2f-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/976cc05a-058b-43d0-95f6-75e6f0425e2f-maas-api-tls\") pod \"976cc05a-058b-43d0-95f6-75e6f0425e2f\" (UID: \"976cc05a-058b-43d0-95f6-75e6f0425e2f\") "
Oct 30 22:07:58.489596 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:07:58.489544    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/976cc05a-058b-43d0-95f6-75e6f0425e2f-metrics-certs\" (UniqueName: \"kubernetes.io/secret/976cc05a-058b-43d0-95f6-75e6f0425e2f-metrics-certs\") pod \"976cc05a-058b-43d0-95f6-75e6f0425e2f\" (UID: \"976cc05a-058b-43d0-95f6-75e6f0425e2f\") "
Oct 30 22:07:58.489748 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:07:58.489719    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/976cc05a-058b-43d0-95f6-75e6f0425e2f-openshift-service-ca" pod "976cc05a-058b-43d0-95f6-75e6f0425e2f" (UID: "976cc05a-058b-43d0-95f6-75e6f0425e2f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:07:58.498881 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:07:58.498422    4473 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/976cc05a-058b-43d0-95f6-75e6f0425e2f-openshift-service-ca\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:07:58.502456 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:07:58.502412    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/976cc05a-058b-43d0-95f6-75e6f0425e2f-maas-api-tls" pod "976cc05a-058b-43d0-95f6-75e6f0425e2f" (UID: "976cc05a-058b-43d0-95f6-75e6f0425e2f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:07:58.526482 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:07:58.526439    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/976cc05a-058b-43d0-95f6-75e6f0425e2f-kube-api-access-n7bdw" pod "976cc05a-058b-43d0-95f6-75e6f0425e2f" (UID: "976cc05a-058b-43d0-95f6-75e6f0425e2f"). InnerVolumeSpecName "kube-api-access-n7bdw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:07:58.527114 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:07:58.527083    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/976cc05a-058b-43d0-95f6-75e6f0425e2f-metrics-certs" pod "976cc05a-058b-43d0-95f6-75e6f0425e2f" (UID: "976cc05a-058b-43d0-95f6-75e6f0425e2f"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:07:58.599819 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:07:58.599785    4473 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n7bdw\" (UniqueName: \"kubernetes.io/projected/976cc05a-058b-43d0-95f6-75e6f0425e2f-kube-api-access-n7bdw\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:07:58.599819 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:07:58.599818    4473 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/976cc05a-058b-43d0-95f6-75e6f0425e2f-maas-api-tls\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:07:58.599819 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:07:58.599828    4473 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/976cc05a-058b-43d0-95f6-75e6f0425e2f-metrics-certs\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:07:59.372330 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:07:59.372304    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1d71f464-7b656bc7b4-pqlb6"
Oct 30 22:07:59.542161 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:07:59.542119    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-1d71f464-7b656bc7b4-pqlb6"]
Oct 30 22:07:59.546593 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:07:59.546564    4473 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-1d71f464-7b656bc7b4-pqlb6"]
Oct 30 22:08:00.222442 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:00.222403    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-1d71f464-openshift-default-55f9d6d6bc-c4fqx"]
Oct 30 22:08:00.222687 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:00.222647    4473 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-1d71f464-openshift-default-55f9d6d6bc-c4fqx" podUID="071132ee-06b1-4b98-9905-6afd31e4289b" containerName="istio-proxy" containerID="cri-o://5031517fce58bf8bdcb380d35c8c386e94aac5fb6392c8ef1ca19d79233e723f" gracePeriod=30
Oct 30 22:08:00.361495 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:00.361453    4473 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="976cc05a-058b-43d0-95f6-75e6f0425e2f" path="/var/lib/kubelet/pods/976cc05a-058b-43d0-95f6-75e6f0425e2f/volumes"
Oct 30 22:08:01.755683 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:01.755646    4473 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-90f9e9d3-gw-openshift-default-55569c489-fllqv"]
Oct 30 22:08:01.756472 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:01.756455    4473 cpu_manager.go:414] "containerMap: removing container" podUID="fe4f0d92-1deb-445e-b700-2e62abd9e4e3" containerName="payload-pre-processing"
Oct 30 22:08:01.756517 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:01.756474    4473 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fe4f0d92-1deb-445e-b700-2e62abd9e4e3" containerName="payload-pre-processing"
Oct 30 22:08:01.756517 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:01.756487    4473 cpu_manager.go:414] "containerMap: removing container" podUID="976cc05a-058b-43d0-95f6-75e6f0425e2f" containerName="maas-api"
Oct 30 22:08:01.756517 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:01.756492    4473 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="976cc05a-058b-43d0-95f6-75e6f0425e2f" containerName="maas-api"
Oct 30 22:08:01.756517 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:01.756509    4473 cpu_manager.go:414] "containerMap: removing container" podUID="6f549a4c-acb7-46b6-85bb-65a7dd17a503" containerName="payload-pre-processing"
Oct 30 22:08:01.756517 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:01.756514    4473 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6f549a4c-acb7-46b6-85bb-65a7dd17a503" containerName="payload-pre-processing"
Oct 30 22:08:01.756670 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:01.756629    4473 memory_manager.go:365] "RemoveStaleState removing state" podUID="fe4f0d92-1deb-445e-b700-2e62abd9e4e3" containerName="payload-pre-processing"
Oct 30 22:08:01.756670 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:01.756637    4473 memory_manager.go:365] "RemoveStaleState removing state" podUID="976cc05a-058b-43d0-95f6-75e6f0425e2f" containerName="maas-api"
Oct 30 22:08:01.756670 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:01.756647    4473 memory_manager.go:365] "RemoveStaleState removing state" podUID="6f549a4c-acb7-46b6-85bb-65a7dd17a503" containerName="payload-pre-processing"
Oct 30 22:08:01.759969 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:01.759946    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-90f9e9d3-gw-openshift-default-55569c489-fllqv"
Oct 30 22:08:01.833947 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:01.823401    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-90f9e9d3-gw-openshift-default-dockercfg-v9299\""
Oct 30 22:08:01.833947 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:01.825473    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bad44c70-cd8b-415c-8d9a-86515b14a266-istio-podinfo\") pod \"e2e-migrate-90f9e9d3-gw-openshift-default-55569c489-fllqv\" (UID: \"bad44c70-cd8b-415c-8d9a-86515b14a266\") " pod="openshift-ingress/e2e-migrate-90f9e9d3-gw-openshift-default-55569c489-fllqv"
Oct 30 22:08:01.833947 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:01.825514    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sdkb6\" (UniqueName: \"kubernetes.io/projected/bad44c70-cd8b-415c-8d9a-86515b14a266-kube-api-access-sdkb6\") pod \"e2e-migrate-90f9e9d3-gw-openshift-default-55569c489-fllqv\" (UID: \"bad44c70-cd8b-415c-8d9a-86515b14a266\") " pod="openshift-ingress/e2e-migrate-90f9e9d3-gw-openshift-default-55569c489-fllqv"
Oct 30 22:08:01.833947 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:01.825661    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bad44c70-cd8b-415c-8d9a-86515b14a266-workload-certs\") pod \"e2e-migrate-90f9e9d3-gw-openshift-default-55569c489-fllqv\" (UID: \"bad44c70-cd8b-415c-8d9a-86515b14a266\") " pod="openshift-ingress/e2e-migrate-90f9e9d3-gw-openshift-default-55569c489-fllqv"
Oct 30 22:08:01.833947 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:01.825830    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bad44c70-cd8b-415c-8d9a-86515b14a266-workload-socket\") pod \"e2e-migrate-90f9e9d3-gw-openshift-default-55569c489-fllqv\" (UID: \"bad44c70-cd8b-415c-8d9a-86515b14a266\") " pod="openshift-ingress/e2e-migrate-90f9e9d3-gw-openshift-default-55569c489-fllqv"
Oct 30 22:08:01.833947 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:01.825862    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bad44c70-cd8b-415c-8d9a-86515b14a266-istio-data\") pod \"e2e-migrate-90f9e9d3-gw-openshift-default-55569c489-fllqv\" (UID: \"bad44c70-cd8b-415c-8d9a-86515b14a266\") " pod="openshift-ingress/e2e-migrate-90f9e9d3-gw-openshift-default-55569c489-fllqv"
Oct 30 22:08:01.833947 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:01.825939    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bad44c70-cd8b-415c-8d9a-86515b14a266-istio-token\") pod \"e2e-migrate-90f9e9d3-gw-openshift-default-55569c489-fllqv\" (UID: \"bad44c70-cd8b-415c-8d9a-86515b14a266\") " pod="openshift-ingress/e2e-migrate-90f9e9d3-gw-openshift-default-55569c489-fllqv"
Oct 30 22:08:01.833947 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:01.825994    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bad44c70-cd8b-415c-8d9a-86515b14a266-istio-envoy\") pod \"e2e-migrate-90f9e9d3-gw-openshift-default-55569c489-fllqv\" (UID: \"bad44c70-cd8b-415c-8d9a-86515b14a266\") " pod="openshift-ingress/e2e-migrate-90f9e9d3-gw-openshift-default-55569c489-fllqv"
Oct 30 22:08:01.833947 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:01.826014    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bad44c70-cd8b-415c-8d9a-86515b14a266-istiod-ca-cert\") pod \"e2e-migrate-90f9e9d3-gw-openshift-default-55569c489-fllqv\" (UID: \"bad44c70-cd8b-415c-8d9a-86515b14a266\") " pod="openshift-ingress/e2e-migrate-90f9e9d3-gw-openshift-default-55569c489-fllqv"
Oct 30 22:08:01.833947 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:01.826049    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bad44c70-cd8b-415c-8d9a-86515b14a266-credential-socket\") pod \"e2e-migrate-90f9e9d3-gw-openshift-default-55569c489-fllqv\" (UID: \"bad44c70-cd8b-415c-8d9a-86515b14a266\") " pod="openshift-ingress/e2e-migrate-90f9e9d3-gw-openshift-default-55569c489-fllqv"
Oct 30 22:08:02.142019 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:02.134480    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-90f9e9d3-gw-openshift-default-55569c489-fllqv"]
Oct 30 22:08:02.207946 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:02.202313    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bad44c70-cd8b-415c-8d9a-86515b14a266-istio-podinfo\") pod \"e2e-migrate-90f9e9d3-gw-openshift-default-55569c489-fllqv\" (UID: \"bad44c70-cd8b-415c-8d9a-86515b14a266\") " pod="openshift-ingress/e2e-migrate-90f9e9d3-gw-openshift-default-55569c489-fllqv"
Oct 30 22:08:02.207946 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:02.202365    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sdkb6\" (UniqueName: \"kubernetes.io/projected/bad44c70-cd8b-415c-8d9a-86515b14a266-kube-api-access-sdkb6\") pod \"e2e-migrate-90f9e9d3-gw-openshift-default-55569c489-fllqv\" (UID: \"bad44c70-cd8b-415c-8d9a-86515b14a266\") " pod="openshift-ingress/e2e-migrate-90f9e9d3-gw-openshift-default-55569c489-fllqv"
Oct 30 22:08:02.207946 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:02.202533    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bad44c70-cd8b-415c-8d9a-86515b14a266-workload-certs\") pod \"e2e-migrate-90f9e9d3-gw-openshift-default-55569c489-fllqv\" (UID: \"bad44c70-cd8b-415c-8d9a-86515b14a266\") " pod="openshift-ingress/e2e-migrate-90f9e9d3-gw-openshift-default-55569c489-fllqv"
Oct 30 22:08:02.207946 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:02.202719    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bad44c70-cd8b-415c-8d9a-86515b14a266-workload-socket\") pod \"e2e-migrate-90f9e9d3-gw-openshift-default-55569c489-fllqv\" (UID: \"bad44c70-cd8b-415c-8d9a-86515b14a266\") " pod="openshift-ingress/e2e-migrate-90f9e9d3-gw-openshift-default-55569c489-fllqv"
Oct 30 22:08:02.207946 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:02.202749    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bad44c70-cd8b-415c-8d9a-86515b14a266-istio-data\") pod \"e2e-migrate-90f9e9d3-gw-openshift-default-55569c489-fllqv\" (UID: \"bad44c70-cd8b-415c-8d9a-86515b14a266\") " pod="openshift-ingress/e2e-migrate-90f9e9d3-gw-openshift-default-55569c489-fllqv"
Oct 30 22:08:02.207946 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:02.202805    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bad44c70-cd8b-415c-8d9a-86515b14a266-istio-token\") pod \"e2e-migrate-90f9e9d3-gw-openshift-default-55569c489-fllqv\" (UID: \"bad44c70-cd8b-415c-8d9a-86515b14a266\") " pod="openshift-ingress/e2e-migrate-90f9e9d3-gw-openshift-default-55569c489-fllqv"
Oct 30 22:08:02.207946 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:02.202848    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bad44c70-cd8b-415c-8d9a-86515b14a266-istio-envoy\") pod \"e2e-migrate-90f9e9d3-gw-openshift-default-55569c489-fllqv\" (UID: \"bad44c70-cd8b-415c-8d9a-86515b14a266\") " pod="openshift-ingress/e2e-migrate-90f9e9d3-gw-openshift-default-55569c489-fllqv"
Oct 30 22:08:02.207946 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:02.202870    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bad44c70-cd8b-415c-8d9a-86515b14a266-istiod-ca-cert\") pod \"e2e-migrate-90f9e9d3-gw-openshift-default-55569c489-fllqv\" (UID: \"bad44c70-cd8b-415c-8d9a-86515b14a266\") " pod="openshift-ingress/e2e-migrate-90f9e9d3-gw-openshift-default-55569c489-fllqv"
Oct 30 22:08:02.207946 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:02.202901    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bad44c70-cd8b-415c-8d9a-86515b14a266-credential-socket\") pod \"e2e-migrate-90f9e9d3-gw-openshift-default-55569c489-fllqv\" (UID: \"bad44c70-cd8b-415c-8d9a-86515b14a266\") " pod="openshift-ingress/e2e-migrate-90f9e9d3-gw-openshift-default-55569c489-fllqv"
Oct 30 22:08:02.207946 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:02.203707    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bad44c70-cd8b-415c-8d9a-86515b14a266-credential-socket\") pod \"e2e-migrate-90f9e9d3-gw-openshift-default-55569c489-fllqv\" (UID: \"bad44c70-cd8b-415c-8d9a-86515b14a266\") " pod="openshift-ingress/e2e-migrate-90f9e9d3-gw-openshift-default-55569c489-fllqv"
Oct 30 22:08:02.207946 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:02.205246    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bad44c70-cd8b-415c-8d9a-86515b14a266-istio-data\") pod \"e2e-migrate-90f9e9d3-gw-openshift-default-55569c489-fllqv\" (UID: \"bad44c70-cd8b-415c-8d9a-86515b14a266\") " pod="openshift-ingress/e2e-migrate-90f9e9d3-gw-openshift-default-55569c489-fllqv"
Oct 30 22:08:02.220945 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:02.217374    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bad44c70-cd8b-415c-8d9a-86515b14a266-workload-socket\") pod \"e2e-migrate-90f9e9d3-gw-openshift-default-55569c489-fllqv\" (UID: \"bad44c70-cd8b-415c-8d9a-86515b14a266\") " pod="openshift-ingress/e2e-migrate-90f9e9d3-gw-openshift-default-55569c489-fllqv"
Oct 30 22:08:02.254197 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:02.229671    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bad44c70-cd8b-415c-8d9a-86515b14a266-istiod-ca-cert\") pod \"e2e-migrate-90f9e9d3-gw-openshift-default-55569c489-fllqv\" (UID: \"bad44c70-cd8b-415c-8d9a-86515b14a266\") " pod="openshift-ingress/e2e-migrate-90f9e9d3-gw-openshift-default-55569c489-fllqv"
Oct 30 22:08:02.254197 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:02.251211    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bad44c70-cd8b-415c-8d9a-86515b14a266-workload-certs\") pod \"e2e-migrate-90f9e9d3-gw-openshift-default-55569c489-fllqv\" (UID: \"bad44c70-cd8b-415c-8d9a-86515b14a266\") " pod="openshift-ingress/e2e-migrate-90f9e9d3-gw-openshift-default-55569c489-fllqv"
Oct 30 22:08:02.288018 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:02.287117    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bad44c70-cd8b-415c-8d9a-86515b14a266-istio-token\") pod \"e2e-migrate-90f9e9d3-gw-openshift-default-55569c489-fllqv\" (UID: \"bad44c70-cd8b-415c-8d9a-86515b14a266\") " pod="openshift-ingress/e2e-migrate-90f9e9d3-gw-openshift-default-55569c489-fllqv"
Oct 30 22:08:02.293833 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:02.293763    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sdkb6\" (UniqueName: \"kubernetes.io/projected/bad44c70-cd8b-415c-8d9a-86515b14a266-kube-api-access-sdkb6\") pod \"e2e-migrate-90f9e9d3-gw-openshift-default-55569c489-fllqv\" (UID: \"bad44c70-cd8b-415c-8d9a-86515b14a266\") " pod="openshift-ingress/e2e-migrate-90f9e9d3-gw-openshift-default-55569c489-fllqv"
Oct 30 22:08:02.327405 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:02.327056    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bad44c70-cd8b-415c-8d9a-86515b14a266-istio-podinfo\") pod \"e2e-migrate-90f9e9d3-gw-openshift-default-55569c489-fllqv\" (UID: \"bad44c70-cd8b-415c-8d9a-86515b14a266\") " pod="openshift-ingress/e2e-migrate-90f9e9d3-gw-openshift-default-55569c489-fllqv"
Oct 30 22:08:02.344049 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:02.338911    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bad44c70-cd8b-415c-8d9a-86515b14a266-istio-envoy\") pod \"e2e-migrate-90f9e9d3-gw-openshift-default-55569c489-fllqv\" (UID: \"bad44c70-cd8b-415c-8d9a-86515b14a266\") " pod="openshift-ingress/e2e-migrate-90f9e9d3-gw-openshift-default-55569c489-fllqv"
Oct 30 22:08:02.433000 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:02.427838    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-90f9e9d3-gw-openshift-default-55569c489-fllqv"
Oct 30 22:08:03.020099 ip-10-0-140-18 kubenswrapper[4473]: W1030 22:08:03.020067    4473 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbad44c70_cd8b_415c_8d9a_86515b14a266.slice/crio-d5b2dc412589cf9faaf48e6975e1b1a7f501d91af6cf4962126e2f667718e1ea WatchSource:0}: Error finding container d5b2dc412589cf9faaf48e6975e1b1a7f501d91af6cf4962126e2f667718e1ea: Status 404 returned error can't find the container with id d5b2dc412589cf9faaf48e6975e1b1a7f501d91af6cf4962126e2f667718e1ea
Oct 30 22:08:03.020723 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:03.020702    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-90f9e9d3-gw-openshift-default-55569c489-fllqv"]
Oct 30 22:08:03.022779 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:03.022743    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:08:03.022884 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:03.022810    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:08:03.022884 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:03.022845    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:08:03.512104 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:03.512064    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-90f9e9d3-gw-openshift-default-55569c489-fllqv" event={"ID":"bad44c70-cd8b-415c-8d9a-86515b14a266","Type":"ContainerStarted","Data":"67155ca335eefca79662fdd7a204683fc189865ba832b1fbc7ad95336c047da9"}
Oct 30 22:08:03.512289 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:03.512128    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-90f9e9d3-gw-openshift-default-55569c489-fllqv" event={"ID":"bad44c70-cd8b-415c-8d9a-86515b14a266","Type":"ContainerStarted","Data":"d5b2dc412589cf9faaf48e6975e1b1a7f501d91af6cf4962126e2f667718e1ea"}
Oct 30 22:08:03.640246 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:03.640194    4473 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-90f9e9d3-gw-openshift-default-55569c489-fllqv" podStartSLOduration=2.640178284 podStartE2EDuration="2.640178284s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:08:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:08:03.63944518 +0000 UTC m=+1529.412813667" watchObservedRunningTime="2026-10-30 22:08:03.640178284 +0000 UTC m=+1529.413546771"
Oct 30 22:08:04.450181 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:04.450140    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-90f9e9d3-gw-openshift-default-55569c489-fllqv"
Oct 30 22:08:04.455772 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:04.455744    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-90f9e9d3-gw-openshift-default-55569c489-fllqv"
Oct 30 22:08:04.520217 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:04.520177    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-90f9e9d3-gw-openshift-default-55569c489-fllqv"
Oct 30 22:08:04.521457 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:04.521432    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-90f9e9d3-gw-openshift-default-55569c489-fllqv"
Oct 30 22:08:04.618032 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:04.617993    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-64086213-799ffc7fbc-jjd6r"]
Oct 30 22:08:04.618233 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:04.618212    4473 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-64086213-799ffc7fbc-jjd6r" podUID="8c51747e-187c-403e-8bfb-98b13d0d9077" containerName="maas-api" containerID="cri-o://308289e6fad1336c9f1504f2c21edb726631162cec05a72b99a794ac4a22c62f" gracePeriod=30
Oct 30 22:08:05.233068 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:05.231809    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-64086213-799ffc7fbc-jjd6r"
Oct 30 22:08:05.277505 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:05.277465    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8c51747e-187c-403e-8bfb-98b13d0d9077-kube-api-access-cfxlc\" (UniqueName: \"kubernetes.io/projected/8c51747e-187c-403e-8bfb-98b13d0d9077-kube-api-access-cfxlc\") pod \"8c51747e-187c-403e-8bfb-98b13d0d9077\" (UID: \"8c51747e-187c-403e-8bfb-98b13d0d9077\") "
Oct 30 22:08:05.280463 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:05.280433    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8c51747e-187c-403e-8bfb-98b13d0d9077-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8c51747e-187c-403e-8bfb-98b13d0d9077-maas-api-tls\") pod \"8c51747e-187c-403e-8bfb-98b13d0d9077\" (UID: \"8c51747e-187c-403e-8bfb-98b13d0d9077\") "
Oct 30 22:08:05.280789 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:05.280769    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8c51747e-187c-403e-8bfb-98b13d0d9077-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8c51747e-187c-403e-8bfb-98b13d0d9077-openshift-service-ca\") pod \"8c51747e-187c-403e-8bfb-98b13d0d9077\" (UID: \"8c51747e-187c-403e-8bfb-98b13d0d9077\") "
Oct 30 22:08:05.281803 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:05.281687    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8c51747e-187c-403e-8bfb-98b13d0d9077-openshift-service-ca" pod "8c51747e-187c-403e-8bfb-98b13d0d9077" (UID: "8c51747e-187c-403e-8bfb-98b13d0d9077"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:08:05.282154 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:05.282132    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8c51747e-187c-403e-8bfb-98b13d0d9077-metrics-certs\" (UniqueName: \"kubernetes.io/secret/8c51747e-187c-403e-8bfb-98b13d0d9077-metrics-certs\") pod \"8c51747e-187c-403e-8bfb-98b13d0d9077\" (UID: \"8c51747e-187c-403e-8bfb-98b13d0d9077\") "
Oct 30 22:08:05.283840 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:05.283803    4473 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8c51747e-187c-403e-8bfb-98b13d0d9077-openshift-service-ca\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:08:05.307502 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:05.302393    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8c51747e-187c-403e-8bfb-98b13d0d9077-kube-api-access-cfxlc" pod "8c51747e-187c-403e-8bfb-98b13d0d9077" (UID: "8c51747e-187c-403e-8bfb-98b13d0d9077"). InnerVolumeSpecName "kube-api-access-cfxlc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:08:05.319475 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:05.319440    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8c51747e-187c-403e-8bfb-98b13d0d9077-maas-api-tls" pod "8c51747e-187c-403e-8bfb-98b13d0d9077" (UID: "8c51747e-187c-403e-8bfb-98b13d0d9077"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:08:05.347004 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:05.346072    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8c51747e-187c-403e-8bfb-98b13d0d9077-metrics-certs" pod "8c51747e-187c-403e-8bfb-98b13d0d9077" (UID: "8c51747e-187c-403e-8bfb-98b13d0d9077"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:08:05.398193 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:05.398041    4473 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8c51747e-187c-403e-8bfb-98b13d0d9077-metrics-certs\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:08:05.398193 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:05.398075    4473 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cfxlc\" (UniqueName: \"kubernetes.io/projected/8c51747e-187c-403e-8bfb-98b13d0d9077-kube-api-access-cfxlc\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:08:05.398193 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:05.398091    4473 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8c51747e-187c-403e-8bfb-98b13d0d9077-maas-api-tls\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:08:05.534501 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:05.534466    4473 generic.go:356] "Generic (PLEG): container finished" podID="8c51747e-187c-403e-8bfb-98b13d0d9077" containerID="308289e6fad1336c9f1504f2c21edb726631162cec05a72b99a794ac4a22c62f" exitCode=0
Oct 30 22:08:05.535191 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:05.534531    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-64086213-799ffc7fbc-jjd6r"
Oct 30 22:08:05.535191 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:05.534621    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-64086213-799ffc7fbc-jjd6r" event={"ID":"8c51747e-187c-403e-8bfb-98b13d0d9077","Type":"ContainerDied","Data":"308289e6fad1336c9f1504f2c21edb726631162cec05a72b99a794ac4a22c62f"}
Oct 30 22:08:05.535191 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:05.534673    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-64086213-799ffc7fbc-jjd6r" event={"ID":"8c51747e-187c-403e-8bfb-98b13d0d9077","Type":"ContainerDied","Data":"4a9a34fddc8b64a7c4e38f5e810f05c8f9eee31c85f94e3264b4dfb9dd359289"}
Oct 30 22:08:05.535191 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:05.534707    4473 scope.go:122] "RemoveContainer" containerID="308289e6fad1336c9f1504f2c21edb726631162cec05a72b99a794ac4a22c62f"
Oct 30 22:08:05.540148 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:05.539838    4473 generic.go:356] "Generic (PLEG): container finished" podID="071132ee-06b1-4b98-9905-6afd31e4289b" containerID="5031517fce58bf8bdcb380d35c8c386e94aac5fb6392c8ef1ca19d79233e723f" exitCode=0
Oct 30 22:08:05.541109 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:05.541077    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-1d71f464-openshift-default-55f9d6d6bc-c4fqx" event={"ID":"071132ee-06b1-4b98-9905-6afd31e4289b","Type":"ContainerDied","Data":"5031517fce58bf8bdcb380d35c8c386e94aac5fb6392c8ef1ca19d79233e723f"}
Oct 30 22:08:05.549740 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:05.549715    4473 scope.go:122] "RemoveContainer" containerID="308289e6fad1336c9f1504f2c21edb726631162cec05a72b99a794ac4a22c62f"
Oct 30 22:08:05.550199 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:08:05.550170    4473 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"308289e6fad1336c9f1504f2c21edb726631162cec05a72b99a794ac4a22c62f\": container with ID starting with 308289e6fad1336c9f1504f2c21edb726631162cec05a72b99a794ac4a22c62f not found: ID does not exist" containerID="308289e6fad1336c9f1504f2c21edb726631162cec05a72b99a794ac4a22c62f"
Oct 30 22:08:05.550315 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:05.550216    4473 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"308289e6fad1336c9f1504f2c21edb726631162cec05a72b99a794ac4a22c62f"} err="failed to get container status \"308289e6fad1336c9f1504f2c21edb726631162cec05a72b99a794ac4a22c62f\": rpc error: code = NotFound desc = could not find container \"308289e6fad1336c9f1504f2c21edb726631162cec05a72b99a794ac4a22c62f\": container with ID starting with 308289e6fad1336c9f1504f2c21edb726631162cec05a72b99a794ac4a22c62f not found: ID does not exist"
Oct 30 22:08:05.550315 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:05.550243    4473 scope.go:122] "RemoveContainer" containerID="43a88e25ce00373dfe7cf2bb371405728c2598ac4dfd8589bb08daaf117651ea"
Oct 30 22:08:05.592732 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:05.592702    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-1d71f464-openshift-default-55f9d6d6bc-c4fqx"
Oct 30 22:08:05.702624 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:05.702592    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/071132ee-06b1-4b98-9905-6afd31e4289b-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/071132ee-06b1-4b98-9905-6afd31e4289b-istiod-ca-cert\") pod \"071132ee-06b1-4b98-9905-6afd31e4289b\" (UID: \"071132ee-06b1-4b98-9905-6afd31e4289b\") "
Oct 30 22:08:05.702875 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:05.702856    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/071132ee-06b1-4b98-9905-6afd31e4289b-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/071132ee-06b1-4b98-9905-6afd31e4289b-istio-podinfo\") pod \"071132ee-06b1-4b98-9905-6afd31e4289b\" (UID: \"071132ee-06b1-4b98-9905-6afd31e4289b\") "
Oct 30 22:08:05.703075 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:05.703057    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/071132ee-06b1-4b98-9905-6afd31e4289b-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/071132ee-06b1-4b98-9905-6afd31e4289b-istio-data\") pod \"071132ee-06b1-4b98-9905-6afd31e4289b\" (UID: \"071132ee-06b1-4b98-9905-6afd31e4289b\") "
Oct 30 22:08:05.703168 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:05.703076    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/071132ee-06b1-4b98-9905-6afd31e4289b-istiod-ca-cert" pod "071132ee-06b1-4b98-9905-6afd31e4289b" (UID: "071132ee-06b1-4b98-9905-6afd31e4289b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:08:05.703168 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:05.703123    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/071132ee-06b1-4b98-9905-6afd31e4289b-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/071132ee-06b1-4b98-9905-6afd31e4289b-istio-envoy\") pod \"071132ee-06b1-4b98-9905-6afd31e4289b\" (UID: \"071132ee-06b1-4b98-9905-6afd31e4289b\") "
Oct 30 22:08:05.703272 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:05.703177    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/071132ee-06b1-4b98-9905-6afd31e4289b-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/071132ee-06b1-4b98-9905-6afd31e4289b-workload-socket\") pod \"071132ee-06b1-4b98-9905-6afd31e4289b\" (UID: \"071132ee-06b1-4b98-9905-6afd31e4289b\") "
Oct 30 22:08:05.703341 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:05.703287    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/071132ee-06b1-4b98-9905-6afd31e4289b-istio-token\" (UniqueName: \"kubernetes.io/projected/071132ee-06b1-4b98-9905-6afd31e4289b-istio-token\") pod \"071132ee-06b1-4b98-9905-6afd31e4289b\" (UID: \"071132ee-06b1-4b98-9905-6afd31e4289b\") "
Oct 30 22:08:05.703425 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:05.703392    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/071132ee-06b1-4b98-9905-6afd31e4289b-kube-api-access-cf7r4\" (UniqueName: \"kubernetes.io/projected/071132ee-06b1-4b98-9905-6afd31e4289b-kube-api-access-cf7r4\") pod \"071132ee-06b1-4b98-9905-6afd31e4289b\" (UID: \"071132ee-06b1-4b98-9905-6afd31e4289b\") "
Oct 30 22:08:05.703504 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:05.703477    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/071132ee-06b1-4b98-9905-6afd31e4289b-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/071132ee-06b1-4b98-9905-6afd31e4289b-credential-socket\") pod \"071132ee-06b1-4b98-9905-6afd31e4289b\" (UID: \"071132ee-06b1-4b98-9905-6afd31e4289b\") "
Oct 30 22:08:05.703586 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:05.703554    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/071132ee-06b1-4b98-9905-6afd31e4289b-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/071132ee-06b1-4b98-9905-6afd31e4289b-workload-certs\") pod \"071132ee-06b1-4b98-9905-6afd31e4289b\" (UID: \"071132ee-06b1-4b98-9905-6afd31e4289b\") "
Oct 30 22:08:05.703792 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:05.703476    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/071132ee-06b1-4b98-9905-6afd31e4289b-istio-data" pod "071132ee-06b1-4b98-9905-6afd31e4289b" (UID: "071132ee-06b1-4b98-9905-6afd31e4289b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:08:05.703977 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:05.703738    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/071132ee-06b1-4b98-9905-6afd31e4289b-credential-socket" pod "071132ee-06b1-4b98-9905-6afd31e4289b" (UID: "071132ee-06b1-4b98-9905-6afd31e4289b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:08:05.703977 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:05.703751    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/071132ee-06b1-4b98-9905-6afd31e4289b-workload-socket" pod "071132ee-06b1-4b98-9905-6afd31e4289b" (UID: "071132ee-06b1-4b98-9905-6afd31e4289b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:08:05.704691 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:05.704200    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/071132ee-06b1-4b98-9905-6afd31e4289b-workload-certs" pod "071132ee-06b1-4b98-9905-6afd31e4289b" (UID: "071132ee-06b1-4b98-9905-6afd31e4289b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:08:05.704691 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:05.704627    4473 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/071132ee-06b1-4b98-9905-6afd31e4289b-istio-data\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:08:05.704691 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:05.704642    4473 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/071132ee-06b1-4b98-9905-6afd31e4289b-workload-socket\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:08:05.704691 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:05.704653    4473 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/071132ee-06b1-4b98-9905-6afd31e4289b-credential-socket\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:08:05.704691 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:05.704662    4473 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/071132ee-06b1-4b98-9905-6afd31e4289b-workload-certs\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:08:05.704691 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:05.704672    4473 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/071132ee-06b1-4b98-9905-6afd31e4289b-istiod-ca-cert\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:08:05.706488 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:05.706448    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/071132ee-06b1-4b98-9905-6afd31e4289b-istio-token" pod "071132ee-06b1-4b98-9905-6afd31e4289b" (UID: "071132ee-06b1-4b98-9905-6afd31e4289b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:08:05.706745 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:05.706718    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/071132ee-06b1-4b98-9905-6afd31e4289b-istio-envoy" pod "071132ee-06b1-4b98-9905-6afd31e4289b" (UID: "071132ee-06b1-4b98-9905-6afd31e4289b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:08:05.707635 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:05.707609    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/071132ee-06b1-4b98-9905-6afd31e4289b-istio-podinfo" pod "071132ee-06b1-4b98-9905-6afd31e4289b" (UID: "071132ee-06b1-4b98-9905-6afd31e4289b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:08:05.708005 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:05.707986    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/071132ee-06b1-4b98-9905-6afd31e4289b-kube-api-access-cf7r4" pod "071132ee-06b1-4b98-9905-6afd31e4289b" (UID: "071132ee-06b1-4b98-9905-6afd31e4289b"). InnerVolumeSpecName "kube-api-access-cf7r4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:08:05.709331 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:05.709309    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-64086213-799ffc7fbc-jjd6r"]
Oct 30 22:08:05.713416 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:05.713387    4473 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-64086213-799ffc7fbc-jjd6r"]
Oct 30 22:08:05.806102 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:05.806063    4473 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/071132ee-06b1-4b98-9905-6afd31e4289b-istio-envoy\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:08:05.806102 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:05.806096    4473 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/071132ee-06b1-4b98-9905-6afd31e4289b-istio-token\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:08:05.806102 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:05.806106    4473 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cf7r4\" (UniqueName: \"kubernetes.io/projected/071132ee-06b1-4b98-9905-6afd31e4289b-kube-api-access-cf7r4\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:08:05.806387 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:05.806128    4473 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/071132ee-06b1-4b98-9905-6afd31e4289b-istio-podinfo\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:08:06.359552 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:06.359516    4473 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8c51747e-187c-403e-8bfb-98b13d0d9077" path="/var/lib/kubelet/pods/8c51747e-187c-403e-8bfb-98b13d0d9077/volumes"
Oct 30 22:08:06.550331 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:06.550301    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-1d71f464-openshift-default-55f9d6d6bc-c4fqx"
Oct 30 22:08:06.550776 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:06.550327    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-1d71f464-openshift-default-55f9d6d6bc-c4fqx" event={"ID":"071132ee-06b1-4b98-9905-6afd31e4289b","Type":"ContainerDied","Data":"4bae24d75b35c7070ae00d721f5f0139b8f4f610f482ac3bb2748c95d4c6ddbb"}
Oct 30 22:08:06.550776 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:06.550392    4473 scope.go:122] "RemoveContainer" containerID="5031517fce58bf8bdcb380d35c8c386e94aac5fb6392c8ef1ca19d79233e723f"
Oct 30 22:08:06.774665 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:06.774625    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-1d71f464-openshift-default-55f9d6d6bc-c4fqx"]
Oct 30 22:08:06.780357 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:06.780325    4473 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-1d71f464-openshift-default-55f9d6d6bc-c4fqx"]
Oct 30 22:08:08.359042 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:08.359010    4473 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="071132ee-06b1-4b98-9905-6afd31e4289b" path="/var/lib/kubelet/pods/071132ee-06b1-4b98-9905-6afd31e4289b/volumes"
Oct 30 22:08:08.808378 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:08.808344    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-64086213-openshift-default-7575697f5c-dszmm"]
Oct 30 22:08:08.808606 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:08.808578    4473 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-64086213-openshift-default-7575697f5c-dszmm" podUID="8e2ea507-72cc-48c6-87d8-6f1805fbfd43" containerName="istio-proxy" containerID="cri-o://2f93aff36996f899f826a5c00c05d6a58d7af87fa25802b7dcbc4ac3ac9a97b3" gracePeriod=30
Oct 30 22:08:09.239947 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:09.239219    4473 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-90f9e9d3-6c5b7d7795-bfzhs"]
Oct 30 22:08:09.257004 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:09.253943    4473 cpu_manager.go:414] "containerMap: removing container" podUID="071132ee-06b1-4b98-9905-6afd31e4289b" containerName="istio-proxy"
Oct 30 22:08:09.257004 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:09.253981    4473 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="071132ee-06b1-4b98-9905-6afd31e4289b" containerName="istio-proxy"
Oct 30 22:08:09.257004 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:09.254012    4473 cpu_manager.go:414] "containerMap: removing container" podUID="071132ee-06b1-4b98-9905-6afd31e4289b" containerName="istio-proxy"
Oct 30 22:08:09.257004 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:09.254019    4473 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="071132ee-06b1-4b98-9905-6afd31e4289b" containerName="istio-proxy"
Oct 30 22:08:09.257004 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:09.254053    4473 cpu_manager.go:414] "containerMap: removing container" podUID="8c51747e-187c-403e-8bfb-98b13d0d9077" containerName="maas-api"
Oct 30 22:08:09.257004 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:09.254059    4473 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8c51747e-187c-403e-8bfb-98b13d0d9077" containerName="maas-api"
Oct 30 22:08:09.257004 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:09.254259    4473 memory_manager.go:365] "RemoveStaleState removing state" podUID="8c51747e-187c-403e-8bfb-98b13d0d9077" containerName="maas-api"
Oct 30 22:08:09.257004 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:09.254280    4473 memory_manager.go:365] "RemoveStaleState removing state" podUID="071132ee-06b1-4b98-9905-6afd31e4289b" containerName="istio-proxy"
Oct 30 22:08:09.265007 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:09.258446    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-90f9e9d3-6c5b7d7795-bfzhs"
Oct 30 22:08:09.312805 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:09.311597    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-78trv\" (UniqueName: \"kubernetes.io/projected/a81a41ac-1fa3-4416-b4ac-fb9ee9bee450-kube-api-access-78trv\") pod \"payload-processing-e2e-migrate-90f9e9d3-6c5b7d7795-bfzhs\" (UID: \"a81a41ac-1fa3-4416-b4ac-fb9ee9bee450\") " pod="openshift-ingress/payload-processing-e2e-migrate-90f9e9d3-6c5b7d7795-bfzhs"
Oct 30 22:08:09.312805 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:09.311672    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a81a41ac-1fa3-4416-b4ac-fb9ee9bee450-plugins-config-volume\") pod \"payload-processing-e2e-migrate-90f9e9d3-6c5b7d7795-bfzhs\" (UID: \"a81a41ac-1fa3-4416-b4ac-fb9ee9bee450\") " pod="openshift-ingress/payload-processing-e2e-migrate-90f9e9d3-6c5b7d7795-bfzhs"
Oct 30 22:08:09.313268 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:09.313245    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-90f9e9d3-dockercfg-gqchr\""
Oct 30 22:08:09.315952 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:09.314058    4473 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-90f9e9d3\""
Oct 30 22:08:09.352946 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:09.350240    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-90f9e9d3-6c5b7d7795-bfzhs"]
Oct 30 22:08:09.434946 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:09.429703    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a81a41ac-1fa3-4416-b4ac-fb9ee9bee450-plugins-config-volume\") pod \"payload-processing-e2e-migrate-90f9e9d3-6c5b7d7795-bfzhs\" (UID: \"a81a41ac-1fa3-4416-b4ac-fb9ee9bee450\") " pod="openshift-ingress/payload-processing-e2e-migrate-90f9e9d3-6c5b7d7795-bfzhs"
Oct 30 22:08:09.468954 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:09.463546    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-78trv\" (UniqueName: \"kubernetes.io/projected/a81a41ac-1fa3-4416-b4ac-fb9ee9bee450-kube-api-access-78trv\") pod \"payload-processing-e2e-migrate-90f9e9d3-6c5b7d7795-bfzhs\" (UID: \"a81a41ac-1fa3-4416-b4ac-fb9ee9bee450\") " pod="openshift-ingress/payload-processing-e2e-migrate-90f9e9d3-6c5b7d7795-bfzhs"
Oct 30 22:08:09.468954 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:09.463857    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a81a41ac-1fa3-4416-b4ac-fb9ee9bee450-plugins-config-volume\") pod \"payload-processing-e2e-migrate-90f9e9d3-6c5b7d7795-bfzhs\" (UID: \"a81a41ac-1fa3-4416-b4ac-fb9ee9bee450\") " pod="openshift-ingress/payload-processing-e2e-migrate-90f9e9d3-6c5b7d7795-bfzhs"
Oct 30 22:08:09.738958 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:09.726622    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-78trv\" (UniqueName: \"kubernetes.io/projected/a81a41ac-1fa3-4416-b4ac-fb9ee9bee450-kube-api-access-78trv\") pod \"payload-processing-e2e-migrate-90f9e9d3-6c5b7d7795-bfzhs\" (UID: \"a81a41ac-1fa3-4416-b4ac-fb9ee9bee450\") " pod="openshift-ingress/payload-processing-e2e-migrate-90f9e9d3-6c5b7d7795-bfzhs"
Oct 30 22:08:09.738958 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:09.738397    4473 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-90f9e9d3-b455b57f9-rtwpf"]
Oct 30 22:08:09.755946 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:09.750319    4473 memory_manager.go:365] "RemoveStaleState removing state" podUID="071132ee-06b1-4b98-9905-6afd31e4289b" containerName="istio-proxy"
Oct 30 22:08:09.764033 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:09.759367    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-90f9e9d3-b455b57f9-rtwpf"
Oct 30 22:08:09.832500 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:09.784412    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-90f9e9d3-b455b57f9-rtwpf"]
Oct 30 22:08:10.050021 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:09.927024    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ql2c8\" (UniqueName: \"kubernetes.io/projected/cfa1c7a1-06ad-4ea2-b6b7-fa0a1d04cdc7-kube-api-access-ql2c8\") pod \"payload-pre-processing-e2e-migrate-90f9e9d3-b455b57f9-rtwpf\" (UID: \"cfa1c7a1-06ad-4ea2-b6b7-fa0a1d04cdc7\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-90f9e9d3-b455b57f9-rtwpf"
Oct 30 22:08:10.050765 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:10.050729    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cfa1c7a1-06ad-4ea2-b6b7-fa0a1d04cdc7-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-90f9e9d3-b455b57f9-rtwpf\" (UID: \"cfa1c7a1-06ad-4ea2-b6b7-fa0a1d04cdc7\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-90f9e9d3-b455b57f9-rtwpf"
Oct 30 22:08:10.051218 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:10.032183    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-90f9e9d3-6c5b7d7795-bfzhs"
Oct 30 22:08:10.153969 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:10.143105    4473 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-497924bf-openshift-default-85499cb6f-fw82l"]
Oct 30 22:08:10.165516 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:10.162059    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ql2c8\" (UniqueName: \"kubernetes.io/projected/cfa1c7a1-06ad-4ea2-b6b7-fa0a1d04cdc7-kube-api-access-ql2c8\") pod \"payload-pre-processing-e2e-migrate-90f9e9d3-b455b57f9-rtwpf\" (UID: \"cfa1c7a1-06ad-4ea2-b6b7-fa0a1d04cdc7\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-90f9e9d3-b455b57f9-rtwpf"
Oct 30 22:08:10.165516 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:10.162357    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cfa1c7a1-06ad-4ea2-b6b7-fa0a1d04cdc7-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-90f9e9d3-b455b57f9-rtwpf\" (UID: \"cfa1c7a1-06ad-4ea2-b6b7-fa0a1d04cdc7\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-90f9e9d3-b455b57f9-rtwpf"
Oct 30 22:08:10.165516 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:10.163055    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-497924bf-openshift-default-85499cb6f-fw82l"
Oct 30 22:08:10.165516 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:10.163311    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cfa1c7a1-06ad-4ea2-b6b7-fa0a1d04cdc7-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-90f9e9d3-b455b57f9-rtwpf\" (UID: \"cfa1c7a1-06ad-4ea2-b6b7-fa0a1d04cdc7\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-90f9e9d3-b455b57f9-rtwpf"
Oct 30 22:08:10.193943 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:10.187208    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-497924bf-openshift-default-85499cb6f-fw82l"]
Oct 30 22:08:10.219669 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:10.219628    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-497924bf-openshift-default-dockercfg-s9l7s\""
Oct 30 22:08:10.250631 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:10.250558    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ql2c8\" (UniqueName: \"kubernetes.io/projected/cfa1c7a1-06ad-4ea2-b6b7-fa0a1d04cdc7-kube-api-access-ql2c8\") pod \"payload-pre-processing-e2e-migrate-90f9e9d3-b455b57f9-rtwpf\" (UID: \"cfa1c7a1-06ad-4ea2-b6b7-fa0a1d04cdc7\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-90f9e9d3-b455b57f9-rtwpf"
Oct 30 22:08:10.266482 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:10.266378    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/233e0e6f-3447-4d9c-a443-0fa29a3b4adc-workload-certs\") pod \"e2e-derive-497924bf-openshift-default-85499cb6f-fw82l\" (UID: \"233e0e6f-3447-4d9c-a443-0fa29a3b4adc\") " pod="openshift-ingress/e2e-derive-497924bf-openshift-default-85499cb6f-fw82l"
Oct 30 22:08:10.266791 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:10.266767    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t6sr7\" (UniqueName: \"kubernetes.io/projected/233e0e6f-3447-4d9c-a443-0fa29a3b4adc-kube-api-access-t6sr7\") pod \"e2e-derive-497924bf-openshift-default-85499cb6f-fw82l\" (UID: \"233e0e6f-3447-4d9c-a443-0fa29a3b4adc\") " pod="openshift-ingress/e2e-derive-497924bf-openshift-default-85499cb6f-fw82l"
Oct 30 22:08:10.267123 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:10.267103    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/233e0e6f-3447-4d9c-a443-0fa29a3b4adc-workload-socket\") pod \"e2e-derive-497924bf-openshift-default-85499cb6f-fw82l\" (UID: \"233e0e6f-3447-4d9c-a443-0fa29a3b4adc\") " pod="openshift-ingress/e2e-derive-497924bf-openshift-default-85499cb6f-fw82l"
Oct 30 22:08:10.267345 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:10.267326    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/233e0e6f-3447-4d9c-a443-0fa29a3b4adc-istio-podinfo\") pod \"e2e-derive-497924bf-openshift-default-85499cb6f-fw82l\" (UID: \"233e0e6f-3447-4d9c-a443-0fa29a3b4adc\") " pod="openshift-ingress/e2e-derive-497924bf-openshift-default-85499cb6f-fw82l"
Oct 30 22:08:10.267647 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:10.267624    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/233e0e6f-3447-4d9c-a443-0fa29a3b4adc-istio-data\") pod \"e2e-derive-497924bf-openshift-default-85499cb6f-fw82l\" (UID: \"233e0e6f-3447-4d9c-a443-0fa29a3b4adc\") " pod="openshift-ingress/e2e-derive-497924bf-openshift-default-85499cb6f-fw82l"
Oct 30 22:08:10.268071 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:10.268052    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/233e0e6f-3447-4d9c-a443-0fa29a3b4adc-istiod-ca-cert\") pod \"e2e-derive-497924bf-openshift-default-85499cb6f-fw82l\" (UID: \"233e0e6f-3447-4d9c-a443-0fa29a3b4adc\") " pod="openshift-ingress/e2e-derive-497924bf-openshift-default-85499cb6f-fw82l"
Oct 30 22:08:10.268288 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:10.268270    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/233e0e6f-3447-4d9c-a443-0fa29a3b4adc-credential-socket\") pod \"e2e-derive-497924bf-openshift-default-85499cb6f-fw82l\" (UID: \"233e0e6f-3447-4d9c-a443-0fa29a3b4adc\") " pod="openshift-ingress/e2e-derive-497924bf-openshift-default-85499cb6f-fw82l"
Oct 30 22:08:10.268459 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:10.268443    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/233e0e6f-3447-4d9c-a443-0fa29a3b4adc-istio-envoy\") pod \"e2e-derive-497924bf-openshift-default-85499cb6f-fw82l\" (UID: \"233e0e6f-3447-4d9c-a443-0fa29a3b4adc\") " pod="openshift-ingress/e2e-derive-497924bf-openshift-default-85499cb6f-fw82l"
Oct 30 22:08:10.268716 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:10.268697    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/233e0e6f-3447-4d9c-a443-0fa29a3b4adc-istio-token\") pod \"e2e-derive-497924bf-openshift-default-85499cb6f-fw82l\" (UID: \"233e0e6f-3447-4d9c-a443-0fa29a3b4adc\") " pod="openshift-ingress/e2e-derive-497924bf-openshift-default-85499cb6f-fw82l"
Oct 30 22:08:10.423961 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:10.417304    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-90f9e9d3-b455b57f9-rtwpf"
Oct 30 22:08:10.504961 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:10.488896    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/233e0e6f-3447-4d9c-a443-0fa29a3b4adc-istio-data\") pod \"e2e-derive-497924bf-openshift-default-85499cb6f-fw82l\" (UID: \"233e0e6f-3447-4d9c-a443-0fa29a3b4adc\") " pod="openshift-ingress/e2e-derive-497924bf-openshift-default-85499cb6f-fw82l"
Oct 30 22:08:10.504961 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:10.489347    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/233e0e6f-3447-4d9c-a443-0fa29a3b4adc-istiod-ca-cert\") pod \"e2e-derive-497924bf-openshift-default-85499cb6f-fw82l\" (UID: \"233e0e6f-3447-4d9c-a443-0fa29a3b4adc\") " pod="openshift-ingress/e2e-derive-497924bf-openshift-default-85499cb6f-fw82l"
Oct 30 22:08:10.504961 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:10.489399    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/233e0e6f-3447-4d9c-a443-0fa29a3b4adc-credential-socket\") pod \"e2e-derive-497924bf-openshift-default-85499cb6f-fw82l\" (UID: \"233e0e6f-3447-4d9c-a443-0fa29a3b4adc\") " pod="openshift-ingress/e2e-derive-497924bf-openshift-default-85499cb6f-fw82l"
Oct 30 22:08:10.504961 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:10.489424    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/233e0e6f-3447-4d9c-a443-0fa29a3b4adc-istio-envoy\") pod \"e2e-derive-497924bf-openshift-default-85499cb6f-fw82l\" (UID: \"233e0e6f-3447-4d9c-a443-0fa29a3b4adc\") " pod="openshift-ingress/e2e-derive-497924bf-openshift-default-85499cb6f-fw82l"
Oct 30 22:08:10.504961 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:10.489545    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/233e0e6f-3447-4d9c-a443-0fa29a3b4adc-istio-token\") pod \"e2e-derive-497924bf-openshift-default-85499cb6f-fw82l\" (UID: \"233e0e6f-3447-4d9c-a443-0fa29a3b4adc\") " pod="openshift-ingress/e2e-derive-497924bf-openshift-default-85499cb6f-fw82l"
Oct 30 22:08:10.504961 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:10.489610    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/233e0e6f-3447-4d9c-a443-0fa29a3b4adc-workload-certs\") pod \"e2e-derive-497924bf-openshift-default-85499cb6f-fw82l\" (UID: \"233e0e6f-3447-4d9c-a443-0fa29a3b4adc\") " pod="openshift-ingress/e2e-derive-497924bf-openshift-default-85499cb6f-fw82l"
Oct 30 22:08:10.504961 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:10.489661    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t6sr7\" (UniqueName: \"kubernetes.io/projected/233e0e6f-3447-4d9c-a443-0fa29a3b4adc-kube-api-access-t6sr7\") pod \"e2e-derive-497924bf-openshift-default-85499cb6f-fw82l\" (UID: \"233e0e6f-3447-4d9c-a443-0fa29a3b4adc\") " pod="openshift-ingress/e2e-derive-497924bf-openshift-default-85499cb6f-fw82l"
Oct 30 22:08:10.504961 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:10.489828    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/233e0e6f-3447-4d9c-a443-0fa29a3b4adc-workload-socket\") pod \"e2e-derive-497924bf-openshift-default-85499cb6f-fw82l\" (UID: \"233e0e6f-3447-4d9c-a443-0fa29a3b4adc\") " pod="openshift-ingress/e2e-derive-497924bf-openshift-default-85499cb6f-fw82l"
Oct 30 22:08:10.504961 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:10.489856    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/233e0e6f-3447-4d9c-a443-0fa29a3b4adc-istio-podinfo\") pod \"e2e-derive-497924bf-openshift-default-85499cb6f-fw82l\" (UID: \"233e0e6f-3447-4d9c-a443-0fa29a3b4adc\") " pod="openshift-ingress/e2e-derive-497924bf-openshift-default-85499cb6f-fw82l"
Oct 30 22:08:10.542941 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:10.511246    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/233e0e6f-3447-4d9c-a443-0fa29a3b4adc-istio-data\") pod \"e2e-derive-497924bf-openshift-default-85499cb6f-fw82l\" (UID: \"233e0e6f-3447-4d9c-a443-0fa29a3b4adc\") " pod="openshift-ingress/e2e-derive-497924bf-openshift-default-85499cb6f-fw82l"
Oct 30 22:08:10.542941 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:10.511864    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/233e0e6f-3447-4d9c-a443-0fa29a3b4adc-istiod-ca-cert\") pod \"e2e-derive-497924bf-openshift-default-85499cb6f-fw82l\" (UID: \"233e0e6f-3447-4d9c-a443-0fa29a3b4adc\") " pod="openshift-ingress/e2e-derive-497924bf-openshift-default-85499cb6f-fw82l"
Oct 30 22:08:10.542941 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:10.512157    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/233e0e6f-3447-4d9c-a443-0fa29a3b4adc-credential-socket\") pod \"e2e-derive-497924bf-openshift-default-85499cb6f-fw82l\" (UID: \"233e0e6f-3447-4d9c-a443-0fa29a3b4adc\") " pod="openshift-ingress/e2e-derive-497924bf-openshift-default-85499cb6f-fw82l"
Oct 30 22:08:10.580998 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:10.559718    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/233e0e6f-3447-4d9c-a443-0fa29a3b4adc-workload-certs\") pod \"e2e-derive-497924bf-openshift-default-85499cb6f-fw82l\" (UID: \"233e0e6f-3447-4d9c-a443-0fa29a3b4adc\") " pod="openshift-ingress/e2e-derive-497924bf-openshift-default-85499cb6f-fw82l"
Oct 30 22:08:10.580998 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:10.571458    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/233e0e6f-3447-4d9c-a443-0fa29a3b4adc-workload-socket\") pod \"e2e-derive-497924bf-openshift-default-85499cb6f-fw82l\" (UID: \"233e0e6f-3447-4d9c-a443-0fa29a3b4adc\") " pod="openshift-ingress/e2e-derive-497924bf-openshift-default-85499cb6f-fw82l"
Oct 30 22:08:10.670943 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:10.653567    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/233e0e6f-3447-4d9c-a443-0fa29a3b4adc-istio-token\") pod \"e2e-derive-497924bf-openshift-default-85499cb6f-fw82l\" (UID: \"233e0e6f-3447-4d9c-a443-0fa29a3b4adc\") " pod="openshift-ingress/e2e-derive-497924bf-openshift-default-85499cb6f-fw82l"
Oct 30 22:08:10.744845 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:10.744717    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/233e0e6f-3447-4d9c-a443-0fa29a3b4adc-istio-podinfo\") pod \"e2e-derive-497924bf-openshift-default-85499cb6f-fw82l\" (UID: \"233e0e6f-3447-4d9c-a443-0fa29a3b4adc\") " pod="openshift-ingress/e2e-derive-497924bf-openshift-default-85499cb6f-fw82l"
Oct 30 22:08:10.751044 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:10.750966    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/233e0e6f-3447-4d9c-a443-0fa29a3b4adc-istio-envoy\") pod \"e2e-derive-497924bf-openshift-default-85499cb6f-fw82l\" (UID: \"233e0e6f-3447-4d9c-a443-0fa29a3b4adc\") " pod="openshift-ingress/e2e-derive-497924bf-openshift-default-85499cb6f-fw82l"
Oct 30 22:08:10.768947 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:10.752080    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t6sr7\" (UniqueName: \"kubernetes.io/projected/233e0e6f-3447-4d9c-a443-0fa29a3b4adc-kube-api-access-t6sr7\") pod \"e2e-derive-497924bf-openshift-default-85499cb6f-fw82l\" (UID: \"233e0e6f-3447-4d9c-a443-0fa29a3b4adc\") " pod="openshift-ingress/e2e-derive-497924bf-openshift-default-85499cb6f-fw82l"
Oct 30 22:08:10.816714 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:10.815325    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-497924bf-openshift-default-85499cb6f-fw82l"
Oct 30 22:08:10.964467 ip-10-0-140-18 kubenswrapper[4473]: W1030 22:08:10.963909    4473 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda81a41ac_1fa3_4416_b4ac_fb9ee9bee450.slice/crio-97f01cf98eaec24800f0955b94cc0b65d64dca1393e3a9dd35519743848301a9 WatchSource:0}: Error finding container 97f01cf98eaec24800f0955b94cc0b65d64dca1393e3a9dd35519743848301a9: Status 404 returned error can't find the container with id 97f01cf98eaec24800f0955b94cc0b65d64dca1393e3a9dd35519743848301a9
Oct 30 22:08:10.967737 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:10.967240    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-90f9e9d3-6c5b7d7795-bfzhs"]
Oct 30 22:08:11.054651 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:11.054572    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-90f9e9d3-6c5b7d7795-bfzhs" event={"ID":"a81a41ac-1fa3-4416-b4ac-fb9ee9bee450","Type":"ContainerStarted","Data":"97f01cf98eaec24800f0955b94cc0b65d64dca1393e3a9dd35519743848301a9"}
Oct 30 22:08:11.434183 ip-10-0-140-18 kubenswrapper[4473]: W1030 22:08:11.433661    4473 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcfa1c7a1_06ad_4ea2_b6b7_fa0a1d04cdc7.slice/crio-2288d07bbdc9c17aa914fc008566d2240cdddd3b714562d68ebd1a9f2aefded4 WatchSource:0}: Error finding container 2288d07bbdc9c17aa914fc008566d2240cdddd3b714562d68ebd1a9f2aefded4: Status 404 returned error can't find the container with id 2288d07bbdc9c17aa914fc008566d2240cdddd3b714562d68ebd1a9f2aefded4
Oct 30 22:08:11.437810 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:11.437755    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-90f9e9d3-b455b57f9-rtwpf"]
Oct 30 22:08:11.441108 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:11.441066    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:08:11.441218 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:11.441161    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:08:11.441261 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:11.441236    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:08:11.450244 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:11.450201    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-497924bf-openshift-default-85499cb6f-fw82l"]
Oct 30 22:08:12.142700 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:12.134822    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-90f9e9d3-6c5b7d7795-bfzhs" event={"ID":"a81a41ac-1fa3-4416-b4ac-fb9ee9bee450","Type":"ContainerStarted","Data":"8582794f38ff6184fe669d2af76eae39771f3aba98a4c7b4303a91340c4461b0"}
Oct 30 22:08:12.142700 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:12.136159    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-migrate-90f9e9d3-6c5b7d7795-bfzhs"
Oct 30 22:08:12.158003 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:12.157962    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-497924bf-openshift-default-85499cb6f-fw82l" event={"ID":"233e0e6f-3447-4d9c-a443-0fa29a3b4adc","Type":"ContainerStarted","Data":"d155d219b8d5da74829b43cbb01d3e196699214e3ff6d95528e36803559bf89e"}
Oct 30 22:08:12.158281 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:12.158260    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-497924bf-openshift-default-85499cb6f-fw82l" event={"ID":"233e0e6f-3447-4d9c-a443-0fa29a3b4adc","Type":"ContainerStarted","Data":"668f8d3ebb09357385a8a3744bcb5cf84ce0e92a4a79676e74726b34e80395aa"}
Oct 30 22:08:12.177857 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:12.177820    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-90f9e9d3-b455b57f9-rtwpf" event={"ID":"cfa1c7a1-06ad-4ea2-b6b7-fa0a1d04cdc7","Type":"ContainerStarted","Data":"0bd0b505fe21a806ea08b4c592cfe25d6e62748930c32b96a827968977708810"}
Oct 30 22:08:12.178176 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:12.178151    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-90f9e9d3-b455b57f9-rtwpf" event={"ID":"cfa1c7a1-06ad-4ea2-b6b7-fa0a1d04cdc7","Type":"ContainerStarted","Data":"2288d07bbdc9c17aa914fc008566d2240cdddd3b714562d68ebd1a9f2aefded4"}
Oct 30 22:08:12.196948 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:12.179761    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-90f9e9d3-b455b57f9-rtwpf"
Oct 30 22:08:12.278979 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:12.267967    4473 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-migrate-90f9e9d3-6c5b7d7795-bfzhs" podStartSLOduration=3.267897178 podStartE2EDuration="3.267897178s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:08:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:08:12.265569245 +0000 UTC m=+1538.038937732" watchObservedRunningTime="2026-10-30 22:08:12.267897178 +0000 UTC m=+1538.041265670"
Oct 30 22:08:12.415005 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:12.394346    4473 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-497924bf-openshift-default-85499cb6f-fw82l" podStartSLOduration=2.394322481 podStartE2EDuration="2.394322481s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:08:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:08:12.391471373 +0000 UTC m=+1538.164839862" watchObservedRunningTime="2026-10-30 22:08:12.394322481 +0000 UTC m=+1538.167690969"
Oct 30 22:08:12.659096 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:12.652519    4473 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-migrate-90f9e9d3-b455b57f9-rtwpf" podStartSLOduration=3.652494206 podStartE2EDuration="3.652494206s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:08:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:08:12.60379167 +0000 UTC m=+1538.377160158" watchObservedRunningTime="2026-10-30 22:08:12.652494206 +0000 UTC m=+1538.425862695"
Oct 30 22:08:12.888083 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:12.864019    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-497924bf-openshift-default-85499cb6f-fw82l"
Oct 30 22:08:12.948326 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:12.948290    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-497924bf-openshift-default-85499cb6f-fw82l"
Oct 30 22:08:13.218690 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:13.218601    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-497924bf-openshift-default-85499cb6f-fw82l"
Oct 30 22:08:13.230072 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:13.230043    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-497924bf-openshift-default-85499cb6f-fw82l"
Oct 30 22:08:14.239220 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:14.239168    4473 generic.go:356] "Generic (PLEG): container finished" podID="8e2ea507-72cc-48c6-87d8-6f1805fbfd43" containerID="2f93aff36996f899f826a5c00c05d6a58d7af87fa25802b7dcbc4ac3ac9a97b3" exitCode=0
Oct 30 22:08:14.241134 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:14.241086    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-64086213-openshift-default-7575697f5c-dszmm" event={"ID":"8e2ea507-72cc-48c6-87d8-6f1805fbfd43","Type":"ContainerDied","Data":"2f93aff36996f899f826a5c00c05d6a58d7af87fa25802b7dcbc4ac3ac9a97b3"}
Oct 30 22:08:14.244066 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:14.244029    4473 scope.go:122] "RemoveContainer" containerID="1f22a165891cfc9cc1f2a87cb106cd84e60b3f6cd65674c726c8eada468802bf"
Oct 30 22:08:14.440822 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:14.440790    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-64086213-openshift-default-7575697f5c-dszmm"
Oct 30 22:08:14.613363 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:14.612020    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8e2ea507-72cc-48c6-87d8-6f1805fbfd43-istio-token\" (UniqueName: \"kubernetes.io/projected/8e2ea507-72cc-48c6-87d8-6f1805fbfd43-istio-token\") pod \"8e2ea507-72cc-48c6-87d8-6f1805fbfd43\" (UID: \"8e2ea507-72cc-48c6-87d8-6f1805fbfd43\") "
Oct 30 22:08:14.613363 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:14.612504    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/8e2ea507-72cc-48c6-87d8-6f1805fbfd43-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8e2ea507-72cc-48c6-87d8-6f1805fbfd43-istio-podinfo\") pod \"8e2ea507-72cc-48c6-87d8-6f1805fbfd43\" (UID: \"8e2ea507-72cc-48c6-87d8-6f1805fbfd43\") "
Oct 30 22:08:14.613363 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:14.612593    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8e2ea507-72cc-48c6-87d8-6f1805fbfd43-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8e2ea507-72cc-48c6-87d8-6f1805fbfd43-istio-envoy\") pod \"8e2ea507-72cc-48c6-87d8-6f1805fbfd43\" (UID: \"8e2ea507-72cc-48c6-87d8-6f1805fbfd43\") "
Oct 30 22:08:14.613363 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:14.612764    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8e2ea507-72cc-48c6-87d8-6f1805fbfd43-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8e2ea507-72cc-48c6-87d8-6f1805fbfd43-credential-socket\") pod \"8e2ea507-72cc-48c6-87d8-6f1805fbfd43\" (UID: \"8e2ea507-72cc-48c6-87d8-6f1805fbfd43\") "
Oct 30 22:08:14.613363 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:14.612822    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8e2ea507-72cc-48c6-87d8-6f1805fbfd43-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8e2ea507-72cc-48c6-87d8-6f1805fbfd43-workload-certs\") pod \"8e2ea507-72cc-48c6-87d8-6f1805fbfd43\" (UID: \"8e2ea507-72cc-48c6-87d8-6f1805fbfd43\") "
Oct 30 22:08:14.613363 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:14.612894    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8e2ea507-72cc-48c6-87d8-6f1805fbfd43-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8e2ea507-72cc-48c6-87d8-6f1805fbfd43-istiod-ca-cert\") pod \"8e2ea507-72cc-48c6-87d8-6f1805fbfd43\" (UID: \"8e2ea507-72cc-48c6-87d8-6f1805fbfd43\") "
Oct 30 22:08:14.613363 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:14.612949    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8e2ea507-72cc-48c6-87d8-6f1805fbfd43-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8e2ea507-72cc-48c6-87d8-6f1805fbfd43-istio-data\") pod \"8e2ea507-72cc-48c6-87d8-6f1805fbfd43\" (UID: \"8e2ea507-72cc-48c6-87d8-6f1805fbfd43\") "
Oct 30 22:08:14.613363 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:14.612984    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8e2ea507-72cc-48c6-87d8-6f1805fbfd43-kube-api-access-sgwrm\" (UniqueName: \"kubernetes.io/projected/8e2ea507-72cc-48c6-87d8-6f1805fbfd43-kube-api-access-sgwrm\") pod \"8e2ea507-72cc-48c6-87d8-6f1805fbfd43\" (UID: \"8e2ea507-72cc-48c6-87d8-6f1805fbfd43\") "
Oct 30 22:08:14.613363 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:14.613148    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8e2ea507-72cc-48c6-87d8-6f1805fbfd43-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8e2ea507-72cc-48c6-87d8-6f1805fbfd43-workload-socket\") pod \"8e2ea507-72cc-48c6-87d8-6f1805fbfd43\" (UID: \"8e2ea507-72cc-48c6-87d8-6f1805fbfd43\") "
Oct 30 22:08:14.613363 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:14.613177    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8e2ea507-72cc-48c6-87d8-6f1805fbfd43-credential-socket" pod "8e2ea507-72cc-48c6-87d8-6f1805fbfd43" (UID: "8e2ea507-72cc-48c6-87d8-6f1805fbfd43"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:08:14.614026 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:14.613434    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8e2ea507-72cc-48c6-87d8-6f1805fbfd43-istio-data" pod "8e2ea507-72cc-48c6-87d8-6f1805fbfd43" (UID: "8e2ea507-72cc-48c6-87d8-6f1805fbfd43"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:08:14.614026 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:14.613494    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8e2ea507-72cc-48c6-87d8-6f1805fbfd43-istiod-ca-cert" pod "8e2ea507-72cc-48c6-87d8-6f1805fbfd43" (UID: "8e2ea507-72cc-48c6-87d8-6f1805fbfd43"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:08:14.614026 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:14.613671    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8e2ea507-72cc-48c6-87d8-6f1805fbfd43-workload-certs" pod "8e2ea507-72cc-48c6-87d8-6f1805fbfd43" (UID: "8e2ea507-72cc-48c6-87d8-6f1805fbfd43"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:08:14.614026 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:14.613721    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8e2ea507-72cc-48c6-87d8-6f1805fbfd43-workload-socket" pod "8e2ea507-72cc-48c6-87d8-6f1805fbfd43" (UID: "8e2ea507-72cc-48c6-87d8-6f1805fbfd43"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:08:14.615051 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:14.614668    4473 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8e2ea507-72cc-48c6-87d8-6f1805fbfd43-credential-socket\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:08:14.617286 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:14.615854    4473 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8e2ea507-72cc-48c6-87d8-6f1805fbfd43-workload-certs\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:08:14.617286 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:14.615882    4473 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8e2ea507-72cc-48c6-87d8-6f1805fbfd43-istiod-ca-cert\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:08:14.617286 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:14.615893    4473 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8e2ea507-72cc-48c6-87d8-6f1805fbfd43-istio-data\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:08:14.617286 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:14.615901    4473 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8e2ea507-72cc-48c6-87d8-6f1805fbfd43-workload-socket\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:08:14.617286 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:14.616374    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8e2ea507-72cc-48c6-87d8-6f1805fbfd43-istio-token" pod "8e2ea507-72cc-48c6-87d8-6f1805fbfd43" (UID: "8e2ea507-72cc-48c6-87d8-6f1805fbfd43"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:08:14.617286 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:14.616483    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8e2ea507-72cc-48c6-87d8-6f1805fbfd43-istio-podinfo" pod "8e2ea507-72cc-48c6-87d8-6f1805fbfd43" (UID: "8e2ea507-72cc-48c6-87d8-6f1805fbfd43"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:08:14.618196 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:14.618169    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8e2ea507-72cc-48c6-87d8-6f1805fbfd43-istio-envoy" pod "8e2ea507-72cc-48c6-87d8-6f1805fbfd43" (UID: "8e2ea507-72cc-48c6-87d8-6f1805fbfd43"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:08:14.618883 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:14.618849    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8e2ea507-72cc-48c6-87d8-6f1805fbfd43-kube-api-access-sgwrm" pod "8e2ea507-72cc-48c6-87d8-6f1805fbfd43" (UID: "8e2ea507-72cc-48c6-87d8-6f1805fbfd43"). InnerVolumeSpecName "kube-api-access-sgwrm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:08:14.717745 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:14.717707    4473 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sgwrm\" (UniqueName: \"kubernetes.io/projected/8e2ea507-72cc-48c6-87d8-6f1805fbfd43-kube-api-access-sgwrm\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:08:14.717745 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:14.717740    4473 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8e2ea507-72cc-48c6-87d8-6f1805fbfd43-istio-token\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:08:14.717745 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:14.717750    4473 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8e2ea507-72cc-48c6-87d8-6f1805fbfd43-istio-podinfo\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:08:14.718048 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:14.717759    4473 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8e2ea507-72cc-48c6-87d8-6f1805fbfd43-istio-envoy\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:08:15.249999 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:15.249967    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-64086213-openshift-default-7575697f5c-dszmm"
Oct 30 22:08:15.250646 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:15.249971    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-64086213-openshift-default-7575697f5c-dszmm" event={"ID":"8e2ea507-72cc-48c6-87d8-6f1805fbfd43","Type":"ContainerDied","Data":"2ae3763fb5b0457db5b6b31dad21d9b2bc041fd18453880905f07a906f578574"}
Oct 30 22:08:15.250646 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:15.250102    4473 scope.go:122] "RemoveContainer" containerID="2f93aff36996f899f826a5c00c05d6a58d7af87fa25802b7dcbc4ac3ac9a97b3"
Oct 30 22:08:15.468454 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:15.468418    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-64086213-openshift-default-7575697f5c-dszmm"]
Oct 30 22:08:15.520747 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:15.520662    4473 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-64086213-openshift-default-7575697f5c-dszmm"]
Oct 30 22:08:16.360497 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:16.360454    4473 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8e2ea507-72cc-48c6-87d8-6f1805fbfd43" path="/var/lib/kubelet/pods/8e2ea507-72cc-48c6-87d8-6f1805fbfd43/volumes"
Oct 30 22:08:16.632446 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:16.632406    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-97895160-openshift-default-5f45687f99-pzfmz"]
Oct 30 22:08:16.632749 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:16.632717    4473 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-97895160-openshift-default-5f45687f99-pzfmz" podUID="e85a3a56-ad99-4c7f-bfbf-c858e78f1dd0" containerName="istio-proxy" containerID="cri-o://c7b5053c3decda8b882ea24f9c7646d6237816c165e5008dc67da0e23bccf3b4" gracePeriod=30
Oct 30 22:08:19.407763 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:19.403787    4473 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-497924bf-6556b64f5f-wgbj5"]
Oct 30 22:08:19.408298 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:19.408249    4473 cpu_manager.go:414] "containerMap: removing container" podUID="8e2ea507-72cc-48c6-87d8-6f1805fbfd43" containerName="istio-proxy"
Oct 30 22:08:19.408298 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:19.408273    4473 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8e2ea507-72cc-48c6-87d8-6f1805fbfd43" containerName="istio-proxy"
Oct 30 22:08:19.424079 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:19.409663    4473 memory_manager.go:365] "RemoveStaleState removing state" podUID="8e2ea507-72cc-48c6-87d8-6f1805fbfd43" containerName="istio-proxy"
Oct 30 22:08:19.424079 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:19.409696    4473 memory_manager.go:365] "RemoveStaleState removing state" podUID="8e2ea507-72cc-48c6-87d8-6f1805fbfd43" containerName="istio-proxy"
Oct 30 22:08:19.444968 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:19.444880    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-497924bf-6556b64f5f-wgbj5"]
Oct 30 22:08:19.446126 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:19.446073    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-497924bf-6556b64f5f-wgbj5"
Oct 30 22:08:19.477881 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:19.477837    4473 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-497924bf-79c65ff565-959fw"]
Oct 30 22:08:19.479764 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:19.479731    4473 cpu_manager.go:414] "containerMap: removing container" podUID="8e2ea507-72cc-48c6-87d8-6f1805fbfd43" containerName="istio-proxy"
Oct 30 22:08:19.479764 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:19.479763    4473 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8e2ea507-72cc-48c6-87d8-6f1805fbfd43" containerName="istio-proxy"
Oct 30 22:08:19.485754 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:19.485196    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-497924bf-79c65ff565-959fw"
Oct 30 22:08:19.503394 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:19.502969    4473 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-497924bf\""
Oct 30 22:08:19.607947 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:19.569383    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fa9ae053-7674-4271-bb36-522b039a565c-plugins-config-volume\") pod \"payload-processing-e2e-derive-497924bf-79c65ff565-959fw\" (UID: \"fa9ae053-7674-4271-bb36-522b039a565c\") " pod="openshift-ingress/payload-processing-e2e-derive-497924bf-79c65ff565-959fw"
Oct 30 22:08:19.607947 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:19.569546    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/61628ae3-0dd6-4c03-b96a-12e065362aa0-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-497924bf-6556b64f5f-wgbj5\" (UID: \"61628ae3-0dd6-4c03-b96a-12e065362aa0\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-497924bf-6556b64f5f-wgbj5"
Oct 30 22:08:19.607947 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:19.569970    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gncd7\" (UniqueName: \"kubernetes.io/projected/fa9ae053-7674-4271-bb36-522b039a565c-kube-api-access-gncd7\") pod \"payload-processing-e2e-derive-497924bf-79c65ff565-959fw\" (UID: \"fa9ae053-7674-4271-bb36-522b039a565c\") " pod="openshift-ingress/payload-processing-e2e-derive-497924bf-79c65ff565-959fw"
Oct 30 22:08:19.607947 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:19.570150    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7zh98\" (UniqueName: \"kubernetes.io/projected/61628ae3-0dd6-4c03-b96a-12e065362aa0-kube-api-access-7zh98\") pod \"payload-pre-processing-e2e-derive-497924bf-6556b64f5f-wgbj5\" (UID: \"61628ae3-0dd6-4c03-b96a-12e065362aa0\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-497924bf-6556b64f5f-wgbj5"
Oct 30 22:08:19.681052 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:19.655286    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-497924bf-dockercfg-zst4n\""
Oct 30 22:08:19.746136 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:19.683429    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gncd7\" (UniqueName: \"kubernetes.io/projected/fa9ae053-7674-4271-bb36-522b039a565c-kube-api-access-gncd7\") pod \"payload-processing-e2e-derive-497924bf-79c65ff565-959fw\" (UID: \"fa9ae053-7674-4271-bb36-522b039a565c\") " pod="openshift-ingress/payload-processing-e2e-derive-497924bf-79c65ff565-959fw"
Oct 30 22:08:19.746136 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:19.683593    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7zh98\" (UniqueName: \"kubernetes.io/projected/61628ae3-0dd6-4c03-b96a-12e065362aa0-kube-api-access-7zh98\") pod \"payload-pre-processing-e2e-derive-497924bf-6556b64f5f-wgbj5\" (UID: \"61628ae3-0dd6-4c03-b96a-12e065362aa0\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-497924bf-6556b64f5f-wgbj5"
Oct 30 22:08:19.746136 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:19.683633    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fa9ae053-7674-4271-bb36-522b039a565c-plugins-config-volume\") pod \"payload-processing-e2e-derive-497924bf-79c65ff565-959fw\" (UID: \"fa9ae053-7674-4271-bb36-522b039a565c\") " pod="openshift-ingress/payload-processing-e2e-derive-497924bf-79c65ff565-959fw"
Oct 30 22:08:19.746136 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:19.683733    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/61628ae3-0dd6-4c03-b96a-12e065362aa0-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-497924bf-6556b64f5f-wgbj5\" (UID: \"61628ae3-0dd6-4c03-b96a-12e065362aa0\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-497924bf-6556b64f5f-wgbj5"
Oct 30 22:08:19.746136 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:19.684693    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/61628ae3-0dd6-4c03-b96a-12e065362aa0-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-497924bf-6556b64f5f-wgbj5\" (UID: \"61628ae3-0dd6-4c03-b96a-12e065362aa0\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-497924bf-6556b64f5f-wgbj5"
Oct 30 22:08:19.746136 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:19.687142    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fa9ae053-7674-4271-bb36-522b039a565c-plugins-config-volume\") pod \"payload-processing-e2e-derive-497924bf-79c65ff565-959fw\" (UID: \"fa9ae053-7674-4271-bb36-522b039a565c\") " pod="openshift-ingress/payload-processing-e2e-derive-497924bf-79c65ff565-959fw"
Oct 30 22:08:19.877326 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:19.863427    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7zh98\" (UniqueName: \"kubernetes.io/projected/61628ae3-0dd6-4c03-b96a-12e065362aa0-kube-api-access-7zh98\") pod \"payload-pre-processing-e2e-derive-497924bf-6556b64f5f-wgbj5\" (UID: \"61628ae3-0dd6-4c03-b96a-12e065362aa0\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-497924bf-6556b64f5f-wgbj5"
Oct 30 22:08:19.877326 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:19.863506    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-497924bf-79c65ff565-959fw"]
Oct 30 22:08:19.894146 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:19.891651    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gncd7\" (UniqueName: \"kubernetes.io/projected/fa9ae053-7674-4271-bb36-522b039a565c-kube-api-access-gncd7\") pod \"payload-processing-e2e-derive-497924bf-79c65ff565-959fw\" (UID: \"fa9ae053-7674-4271-bb36-522b039a565c\") " pod="openshift-ingress/payload-processing-e2e-derive-497924bf-79c65ff565-959fw"
Oct 30 22:08:20.084556 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:20.084459    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-497924bf-6556b64f5f-wgbj5"
Oct 30 22:08:20.118436 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:20.118402    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-497924bf-79c65ff565-959fw"
Oct 30 22:08:20.326830 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:08:20.326790    4473 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fncvg" containerName="extractor"
Oct 30 22:08:20.516021 ip-10-0-140-18 kubenswrapper[4473]: W1030 22:08:20.515984    4473 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod61628ae3_0dd6_4c03_b96a_12e065362aa0.slice/crio-d16f37d415078f5a32481093d33799baab4d439665b779600c499ba03c659165 WatchSource:0}: Error finding container d16f37d415078f5a32481093d33799baab4d439665b779600c499ba03c659165: Status 404 returned error can't find the container with id d16f37d415078f5a32481093d33799baab4d439665b779600c499ba03c659165
Oct 30 22:08:20.517349 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:20.517323    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-497924bf-6556b64f5f-wgbj5"]
Oct 30 22:08:20.536038 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:20.535881    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-497924bf-79c65ff565-959fw"]
Oct 30 22:08:20.536121 ip-10-0-140-18 kubenswrapper[4473]: W1030 22:08:20.536084    4473 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa9ae053_7674_4271_bb36_522b039a565c.slice/crio-f14ed69a5cd7ea0f6bb9e1d484688e5934c2f29626874eb6067ee316c65e08ae WatchSource:0}: Error finding container f14ed69a5cd7ea0f6bb9e1d484688e5934c2f29626874eb6067ee316c65e08ae: Status 404 returned error can't find the container with id f14ed69a5cd7ea0f6bb9e1d484688e5934c2f29626874eb6067ee316c65e08ae
Oct 30 22:08:21.231997 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:21.231965    4473 patch_prober.go:29] interesting pod/e2e-ait-97895160-openshift-default-5f45687f99-pzfmz container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 22:08:21.232184 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:21.232020    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-97895160-openshift-default-5f45687f99-pzfmz" podUID="e85a3a56-ad99-4c7f-bfbf-c858e78f1dd0" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 22:08:21.347457 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:21.347423    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-497924bf-79c65ff565-959fw" event={"ID":"fa9ae053-7674-4271-bb36-522b039a565c","Type":"ContainerStarted","Data":"f3889605b913bca40af0016a027bab08093dd9dcf236df8a9f17b9c3988f542a"}
Oct 30 22:08:21.347649 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:21.347479    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-497924bf-79c65ff565-959fw" event={"ID":"fa9ae053-7674-4271-bb36-522b039a565c","Type":"ContainerStarted","Data":"f14ed69a5cd7ea0f6bb9e1d484688e5934c2f29626874eb6067ee316c65e08ae"}
Oct 30 22:08:21.347649 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:21.347627    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-derive-497924bf-79c65ff565-959fw"
Oct 30 22:08:21.349339 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:21.349311    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-497924bf-6556b64f5f-wgbj5" event={"ID":"61628ae3-0dd6-4c03-b96a-12e065362aa0","Type":"ContainerStarted","Data":"74220c8de756621952d8e5fa471e990c7a935fbc00837590a8c66f12a70b0a9b"}
Oct 30 22:08:21.349477 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:21.349352    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-497924bf-6556b64f5f-wgbj5" event={"ID":"61628ae3-0dd6-4c03-b96a-12e065362aa0","Type":"ContainerStarted","Data":"d16f37d415078f5a32481093d33799baab4d439665b779600c499ba03c659165"}
Oct 30 22:08:21.349740 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:21.349720    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-497924bf-6556b64f5f-wgbj5"
Oct 30 22:08:21.468439 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:21.468391    4473 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-derive-497924bf-79c65ff565-959fw" podStartSLOduration=2.468375839 podStartE2EDuration="2.468375839s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:08:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:08:21.465973222 +0000 UTC m=+1547.239341711" watchObservedRunningTime="2026-10-30 22:08:21.468375839 +0000 UTC m=+1547.241744319"
Oct 30 22:08:21.618054 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:21.616510    4473 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-derive-497924bf-6556b64f5f-wgbj5" podStartSLOduration=2.61647525 podStartE2EDuration="2.61647525s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:08:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:08:21.616252378 +0000 UTC m=+1547.389620866" watchObservedRunningTime="2026-10-30 22:08:21.61647525 +0000 UTC m=+1547.389843740"
Oct 30 22:08:22.039777 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:22.039748    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-97895160-openshift-default-5f45687f99-pzfmz"
Oct 30 22:08:22.206297 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:22.206195    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e85a3a56-ad99-4c7f-bfbf-c858e78f1dd0-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e85a3a56-ad99-4c7f-bfbf-c858e78f1dd0-istiod-ca-cert\") pod \"e85a3a56-ad99-4c7f-bfbf-c858e78f1dd0\" (UID: \"e85a3a56-ad99-4c7f-bfbf-c858e78f1dd0\") "
Oct 30 22:08:22.206297 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:22.206252    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e85a3a56-ad99-4c7f-bfbf-c858e78f1dd0-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e85a3a56-ad99-4c7f-bfbf-c858e78f1dd0-workload-certs\") pod \"e85a3a56-ad99-4c7f-bfbf-c858e78f1dd0\" (UID: \"e85a3a56-ad99-4c7f-bfbf-c858e78f1dd0\") "
Oct 30 22:08:22.206834 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:22.206311    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/e85a3a56-ad99-4c7f-bfbf-c858e78f1dd0-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e85a3a56-ad99-4c7f-bfbf-c858e78f1dd0-istio-podinfo\") pod \"e85a3a56-ad99-4c7f-bfbf-c858e78f1dd0\" (UID: \"e85a3a56-ad99-4c7f-bfbf-c858e78f1dd0\") "
Oct 30 22:08:22.206834 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:22.206464    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e85a3a56-ad99-4c7f-bfbf-c858e78f1dd0-istio-token\" (UniqueName: \"kubernetes.io/projected/e85a3a56-ad99-4c7f-bfbf-c858e78f1dd0-istio-token\") pod \"e85a3a56-ad99-4c7f-bfbf-c858e78f1dd0\" (UID: \"e85a3a56-ad99-4c7f-bfbf-c858e78f1dd0\") "
Oct 30 22:08:22.206834 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:22.206510    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e85a3a56-ad99-4c7f-bfbf-c858e78f1dd0-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e85a3a56-ad99-4c7f-bfbf-c858e78f1dd0-istio-envoy\") pod \"e85a3a56-ad99-4c7f-bfbf-c858e78f1dd0\" (UID: \"e85a3a56-ad99-4c7f-bfbf-c858e78f1dd0\") "
Oct 30 22:08:22.206834 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:22.206586    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e85a3a56-ad99-4c7f-bfbf-c858e78f1dd0-workload-certs" pod "e85a3a56-ad99-4c7f-bfbf-c858e78f1dd0" (UID: "e85a3a56-ad99-4c7f-bfbf-c858e78f1dd0"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:08:22.206834 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:22.206647    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e85a3a56-ad99-4c7f-bfbf-c858e78f1dd0-istiod-ca-cert" pod "e85a3a56-ad99-4c7f-bfbf-c858e78f1dd0" (UID: "e85a3a56-ad99-4c7f-bfbf-c858e78f1dd0"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:08:22.206834 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:22.206751    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e85a3a56-ad99-4c7f-bfbf-c858e78f1dd0-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e85a3a56-ad99-4c7f-bfbf-c858e78f1dd0-workload-socket\") pod \"e85a3a56-ad99-4c7f-bfbf-c858e78f1dd0\" (UID: \"e85a3a56-ad99-4c7f-bfbf-c858e78f1dd0\") "
Oct 30 22:08:22.207311 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:22.207022    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e85a3a56-ad99-4c7f-bfbf-c858e78f1dd0-kube-api-access-lzwpp\" (UniqueName: \"kubernetes.io/projected/e85a3a56-ad99-4c7f-bfbf-c858e78f1dd0-kube-api-access-lzwpp\") pod \"e85a3a56-ad99-4c7f-bfbf-c858e78f1dd0\" (UID: \"e85a3a56-ad99-4c7f-bfbf-c858e78f1dd0\") "
Oct 30 22:08:22.207311 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:22.207107    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e85a3a56-ad99-4c7f-bfbf-c858e78f1dd0-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e85a3a56-ad99-4c7f-bfbf-c858e78f1dd0-credential-socket\") pod \"e85a3a56-ad99-4c7f-bfbf-c858e78f1dd0\" (UID: \"e85a3a56-ad99-4c7f-bfbf-c858e78f1dd0\") "
Oct 30 22:08:22.207311 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:22.207231    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e85a3a56-ad99-4c7f-bfbf-c858e78f1dd0-workload-socket" pod "e85a3a56-ad99-4c7f-bfbf-c858e78f1dd0" (UID: "e85a3a56-ad99-4c7f-bfbf-c858e78f1dd0"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:08:22.207515 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:22.207343    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e85a3a56-ad99-4c7f-bfbf-c858e78f1dd0-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e85a3a56-ad99-4c7f-bfbf-c858e78f1dd0-istio-data\") pod \"e85a3a56-ad99-4c7f-bfbf-c858e78f1dd0\" (UID: \"e85a3a56-ad99-4c7f-bfbf-c858e78f1dd0\") "
Oct 30 22:08:22.207583 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:22.207517    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e85a3a56-ad99-4c7f-bfbf-c858e78f1dd0-credential-socket" pod "e85a3a56-ad99-4c7f-bfbf-c858e78f1dd0" (UID: "e85a3a56-ad99-4c7f-bfbf-c858e78f1dd0"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:08:22.207583 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:22.207573    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e85a3a56-ad99-4c7f-bfbf-c858e78f1dd0-istio-data" pod "e85a3a56-ad99-4c7f-bfbf-c858e78f1dd0" (UID: "e85a3a56-ad99-4c7f-bfbf-c858e78f1dd0"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:08:22.210299 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:22.209162    4473 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e85a3a56-ad99-4c7f-bfbf-c858e78f1dd0-istio-data\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:08:22.210299 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:22.209187    4473 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e85a3a56-ad99-4c7f-bfbf-c858e78f1dd0-istiod-ca-cert\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:08:22.210299 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:22.209204    4473 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e85a3a56-ad99-4c7f-bfbf-c858e78f1dd0-workload-certs\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:08:22.210299 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:22.209220    4473 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e85a3a56-ad99-4c7f-bfbf-c858e78f1dd0-workload-socket\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:08:22.210299 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:22.209235    4473 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e85a3a56-ad99-4c7f-bfbf-c858e78f1dd0-credential-socket\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:08:22.210299 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:22.210253    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e85a3a56-ad99-4c7f-bfbf-c858e78f1dd0-istio-token" pod "e85a3a56-ad99-4c7f-bfbf-c858e78f1dd0" (UID: "e85a3a56-ad99-4c7f-bfbf-c858e78f1dd0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:08:22.210636 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:22.210330    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e85a3a56-ad99-4c7f-bfbf-c858e78f1dd0-kube-api-access-lzwpp" pod "e85a3a56-ad99-4c7f-bfbf-c858e78f1dd0" (UID: "e85a3a56-ad99-4c7f-bfbf-c858e78f1dd0"). InnerVolumeSpecName "kube-api-access-lzwpp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:08:22.210970 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:22.210907    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e85a3a56-ad99-4c7f-bfbf-c858e78f1dd0-istio-podinfo" pod "e85a3a56-ad99-4c7f-bfbf-c858e78f1dd0" (UID: "e85a3a56-ad99-4c7f-bfbf-c858e78f1dd0"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:08:22.211508 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:22.211482    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e85a3a56-ad99-4c7f-bfbf-c858e78f1dd0-istio-envoy" pod "e85a3a56-ad99-4c7f-bfbf-c858e78f1dd0" (UID: "e85a3a56-ad99-4c7f-bfbf-c858e78f1dd0"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:08:22.310567 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:22.310519    4473 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e85a3a56-ad99-4c7f-bfbf-c858e78f1dd0-istio-podinfo\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:08:22.310567 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:22.310558    4473 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e85a3a56-ad99-4c7f-bfbf-c858e78f1dd0-istio-token\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:08:22.310567 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:22.310572    4473 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e85a3a56-ad99-4c7f-bfbf-c858e78f1dd0-istio-envoy\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:08:22.310817 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:22.310586    4473 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lzwpp\" (UniqueName: \"kubernetes.io/projected/e85a3a56-ad99-4c7f-bfbf-c858e78f1dd0-kube-api-access-lzwpp\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:08:22.358990 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:22.358911    4473 generic.go:356] "Generic (PLEG): container finished" podID="e85a3a56-ad99-4c7f-bfbf-c858e78f1dd0" containerID="c7b5053c3decda8b882ea24f9c7646d6237816c165e5008dc67da0e23bccf3b4" exitCode=0
Oct 30 22:08:22.359169 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:22.359032    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-97895160-openshift-default-5f45687f99-pzfmz"
Oct 30 22:08:22.360205 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:22.360178    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-97895160-openshift-default-5f45687f99-pzfmz" event={"ID":"e85a3a56-ad99-4c7f-bfbf-c858e78f1dd0","Type":"ContainerDied","Data":"c7b5053c3decda8b882ea24f9c7646d6237816c165e5008dc67da0e23bccf3b4"}
Oct 30 22:08:22.360334 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:22.360221    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-97895160-openshift-default-5f45687f99-pzfmz" event={"ID":"e85a3a56-ad99-4c7f-bfbf-c858e78f1dd0","Type":"ContainerDied","Data":"b6ae8e533d44776034721d79d6a9f7c4948d35d7ce0af8d588c77139e86bedd5"}
Oct 30 22:08:22.360334 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:22.360242    4473 scope.go:122] "RemoveContainer" containerID="c7b5053c3decda8b882ea24f9c7646d6237816c165e5008dc67da0e23bccf3b4"
Oct 30 22:08:22.379561 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:22.379536    4473 scope.go:122] "RemoveContainer" containerID="56b99d959e478fff249b8fa8302ee589f3144a062ebd32d58e9aa8e87605f699"
Oct 30 22:08:22.390508 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:22.390463    4473 scope.go:122] "RemoveContainer" containerID="c7b5053c3decda8b882ea24f9c7646d6237816c165e5008dc67da0e23bccf3b4"
Oct 30 22:08:22.391080 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:08:22.391017    4473 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c7b5053c3decda8b882ea24f9c7646d6237816c165e5008dc67da0e23bccf3b4\": container with ID starting with c7b5053c3decda8b882ea24f9c7646d6237816c165e5008dc67da0e23bccf3b4 not found: ID does not exist" containerID="c7b5053c3decda8b882ea24f9c7646d6237816c165e5008dc67da0e23bccf3b4"
Oct 30 22:08:22.391191 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:22.391106    4473 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c7b5053c3decda8b882ea24f9c7646d6237816c165e5008dc67da0e23bccf3b4"} err="failed to get container status \"c7b5053c3decda8b882ea24f9c7646d6237816c165e5008dc67da0e23bccf3b4\": rpc error: code = NotFound desc = could not find container \"c7b5053c3decda8b882ea24f9c7646d6237816c165e5008dc67da0e23bccf3b4\": container with ID starting with c7b5053c3decda8b882ea24f9c7646d6237816c165e5008dc67da0e23bccf3b4 not found: ID does not exist"
Oct 30 22:08:22.391191 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:22.391148    4473 scope.go:122] "RemoveContainer" containerID="56b99d959e478fff249b8fa8302ee589f3144a062ebd32d58e9aa8e87605f699"
Oct 30 22:08:22.391521 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:08:22.391499    4473 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"56b99d959e478fff249b8fa8302ee589f3144a062ebd32d58e9aa8e87605f699\": container with ID starting with 56b99d959e478fff249b8fa8302ee589f3144a062ebd32d58e9aa8e87605f699 not found: ID does not exist" containerID="56b99d959e478fff249b8fa8302ee589f3144a062ebd32d58e9aa8e87605f699"
Oct 30 22:08:22.391595 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:22.391527    4473 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"56b99d959e478fff249b8fa8302ee589f3144a062ebd32d58e9aa8e87605f699"} err="failed to get container status \"56b99d959e478fff249b8fa8302ee589f3144a062ebd32d58e9aa8e87605f699\": rpc error: code = NotFound desc = could not find container \"56b99d959e478fff249b8fa8302ee589f3144a062ebd32d58e9aa8e87605f699\": container with ID starting with 56b99d959e478fff249b8fa8302ee589f3144a062ebd32d58e9aa8e87605f699 not found: ID does not exist"
Oct 30 22:08:22.535229 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:22.535145    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-97895160-openshift-default-5f45687f99-pzfmz"]
Oct 30 22:08:22.541716 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:22.541683    4473 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-97895160-openshift-default-5f45687f99-pzfmz"]
Oct 30 22:08:24.245079 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:24.245049    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-migrate-90f9e9d3-6c5b7d7795-bfzhs"
Oct 30 22:08:24.246803 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:24.246778    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-90f9e9d3-b455b57f9-rtwpf"
Oct 30 22:08:24.360383 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:24.360348    4473 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e85a3a56-ad99-4c7f-bfbf-c858e78f1dd0" path="/var/lib/kubelet/pods/e85a3a56-ad99-4c7f-bfbf-c858e78f1dd0/volumes"
Oct 30 22:08:32.363203 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:32.363173    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-derive-497924bf-79c65ff565-959fw"
Oct 30 22:08:32.363667 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:32.363246    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-497924bf-6556b64f5f-wgbj5"
Oct 30 22:08:49.652208 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:49.651643    4473 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w2-2a25a4-openshift-default-7bfc77bdbf-9prd7"]
Oct 30 22:08:49.669228 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:49.669194    4473 cpu_manager.go:414] "containerMap: removing container" podUID="e85a3a56-ad99-4c7f-bfbf-c858e78f1dd0" containerName="istio-proxy"
Oct 30 22:08:49.669519 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:49.669493    4473 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e85a3a56-ad99-4c7f-bfbf-c858e78f1dd0" containerName="istio-proxy"
Oct 30 22:08:49.669993 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:49.669973    4473 memory_manager.go:365] "RemoveStaleState removing state" podUID="e85a3a56-ad99-4c7f-bfbf-c858e78f1dd0" containerName="istio-proxy"
Oct 30 22:08:49.670133 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:49.670121    4473 memory_manager.go:365] "RemoveStaleState removing state" podUID="e85a3a56-ad99-4c7f-bfbf-c858e78f1dd0" containerName="istio-proxy"
Oct 30 22:08:49.852358 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:49.850364    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-2a25a4-openshift-default-7bfc77bdbf-9prd7"]
Oct 30 22:08:49.852358 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:49.850493    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-2a25a4-openshift-default-7bfc77bdbf-9prd7"
Oct 30 22:08:49.986344 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:49.913538    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w2-2a25a4-openshift-default-dockercfg-znktr\""
Oct 30 22:08:49.986344 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:49.936991    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0195818-f111-47ca-a386-005ede718851-credential-socket\") pod \"e2e-shared-a-w2-2a25a4-openshift-default-7bfc77bdbf-9prd7\" (UID: \"f0195818-f111-47ca-a386-005ede718851\") " pod="openshift-ingress/e2e-shared-a-w2-2a25a4-openshift-default-7bfc77bdbf-9prd7"
Oct 30 22:08:49.986344 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:49.937116    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f0195818-f111-47ca-a386-005ede718851-istio-data\") pod \"e2e-shared-a-w2-2a25a4-openshift-default-7bfc77bdbf-9prd7\" (UID: \"f0195818-f111-47ca-a386-005ede718851\") " pod="openshift-ingress/e2e-shared-a-w2-2a25a4-openshift-default-7bfc77bdbf-9prd7"
Oct 30 22:08:49.986344 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:49.937202    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f0195818-f111-47ca-a386-005ede718851-istiod-ca-cert\") pod \"e2e-shared-a-w2-2a25a4-openshift-default-7bfc77bdbf-9prd7\" (UID: \"f0195818-f111-47ca-a386-005ede718851\") " pod="openshift-ingress/e2e-shared-a-w2-2a25a4-openshift-default-7bfc77bdbf-9prd7"
Oct 30 22:08:49.986344 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:49.937316    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0195818-f111-47ca-a386-005ede718851-workload-socket\") pod \"e2e-shared-a-w2-2a25a4-openshift-default-7bfc77bdbf-9prd7\" (UID: \"f0195818-f111-47ca-a386-005ede718851\") " pod="openshift-ingress/e2e-shared-a-w2-2a25a4-openshift-default-7bfc77bdbf-9prd7"
Oct 30 22:08:49.986344 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:49.937343    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f0195818-f111-47ca-a386-005ede718851-istio-envoy\") pod \"e2e-shared-a-w2-2a25a4-openshift-default-7bfc77bdbf-9prd7\" (UID: \"f0195818-f111-47ca-a386-005ede718851\") " pod="openshift-ingress/e2e-shared-a-w2-2a25a4-openshift-default-7bfc77bdbf-9prd7"
Oct 30 22:08:49.986344 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:49.937475    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xlzr6\" (UniqueName: \"kubernetes.io/projected/f0195818-f111-47ca-a386-005ede718851-kube-api-access-xlzr6\") pod \"e2e-shared-a-w2-2a25a4-openshift-default-7bfc77bdbf-9prd7\" (UID: \"f0195818-f111-47ca-a386-005ede718851\") " pod="openshift-ingress/e2e-shared-a-w2-2a25a4-openshift-default-7bfc77bdbf-9prd7"
Oct 30 22:08:49.986344 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:49.937530    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f0195818-f111-47ca-a386-005ede718851-istio-token\") pod \"e2e-shared-a-w2-2a25a4-openshift-default-7bfc77bdbf-9prd7\" (UID: \"f0195818-f111-47ca-a386-005ede718851\") " pod="openshift-ingress/e2e-shared-a-w2-2a25a4-openshift-default-7bfc77bdbf-9prd7"
Oct 30 22:08:49.986344 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:49.937620    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f0195818-f111-47ca-a386-005ede718851-workload-certs\") pod \"e2e-shared-a-w2-2a25a4-openshift-default-7bfc77bdbf-9prd7\" (UID: \"f0195818-f111-47ca-a386-005ede718851\") " pod="openshift-ingress/e2e-shared-a-w2-2a25a4-openshift-default-7bfc77bdbf-9prd7"
Oct 30 22:08:49.986344 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:49.937684    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f0195818-f111-47ca-a386-005ede718851-istio-podinfo\") pod \"e2e-shared-a-w2-2a25a4-openshift-default-7bfc77bdbf-9prd7\" (UID: \"f0195818-f111-47ca-a386-005ede718851\") " pod="openshift-ingress/e2e-shared-a-w2-2a25a4-openshift-default-7bfc77bdbf-9prd7"
Oct 30 22:08:50.078943 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:50.070217    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0195818-f111-47ca-a386-005ede718851-workload-socket\") pod \"e2e-shared-a-w2-2a25a4-openshift-default-7bfc77bdbf-9prd7\" (UID: \"f0195818-f111-47ca-a386-005ede718851\") " pod="openshift-ingress/e2e-shared-a-w2-2a25a4-openshift-default-7bfc77bdbf-9prd7"
Oct 30 22:08:50.078943 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:50.070275    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f0195818-f111-47ca-a386-005ede718851-istio-envoy\") pod \"e2e-shared-a-w2-2a25a4-openshift-default-7bfc77bdbf-9prd7\" (UID: \"f0195818-f111-47ca-a386-005ede718851\") " pod="openshift-ingress/e2e-shared-a-w2-2a25a4-openshift-default-7bfc77bdbf-9prd7"
Oct 30 22:08:50.078943 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:50.070394    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xlzr6\" (UniqueName: \"kubernetes.io/projected/f0195818-f111-47ca-a386-005ede718851-kube-api-access-xlzr6\") pod \"e2e-shared-a-w2-2a25a4-openshift-default-7bfc77bdbf-9prd7\" (UID: \"f0195818-f111-47ca-a386-005ede718851\") " pod="openshift-ingress/e2e-shared-a-w2-2a25a4-openshift-default-7bfc77bdbf-9prd7"
Oct 30 22:08:50.078943 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:50.070450    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f0195818-f111-47ca-a386-005ede718851-istio-token\") pod \"e2e-shared-a-w2-2a25a4-openshift-default-7bfc77bdbf-9prd7\" (UID: \"f0195818-f111-47ca-a386-005ede718851\") " pod="openshift-ingress/e2e-shared-a-w2-2a25a4-openshift-default-7bfc77bdbf-9prd7"
Oct 30 22:08:50.078943 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:50.070528    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f0195818-f111-47ca-a386-005ede718851-workload-certs\") pod \"e2e-shared-a-w2-2a25a4-openshift-default-7bfc77bdbf-9prd7\" (UID: \"f0195818-f111-47ca-a386-005ede718851\") " pod="openshift-ingress/e2e-shared-a-w2-2a25a4-openshift-default-7bfc77bdbf-9prd7"
Oct 30 22:08:50.078943 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:50.070596    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f0195818-f111-47ca-a386-005ede718851-istio-podinfo\") pod \"e2e-shared-a-w2-2a25a4-openshift-default-7bfc77bdbf-9prd7\" (UID: \"f0195818-f111-47ca-a386-005ede718851\") " pod="openshift-ingress/e2e-shared-a-w2-2a25a4-openshift-default-7bfc77bdbf-9prd7"
Oct 30 22:08:50.078943 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:50.070667    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0195818-f111-47ca-a386-005ede718851-credential-socket\") pod \"e2e-shared-a-w2-2a25a4-openshift-default-7bfc77bdbf-9prd7\" (UID: \"f0195818-f111-47ca-a386-005ede718851\") " pod="openshift-ingress/e2e-shared-a-w2-2a25a4-openshift-default-7bfc77bdbf-9prd7"
Oct 30 22:08:50.078943 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:50.070755    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f0195818-f111-47ca-a386-005ede718851-istio-data\") pod \"e2e-shared-a-w2-2a25a4-openshift-default-7bfc77bdbf-9prd7\" (UID: \"f0195818-f111-47ca-a386-005ede718851\") " pod="openshift-ingress/e2e-shared-a-w2-2a25a4-openshift-default-7bfc77bdbf-9prd7"
Oct 30 22:08:50.078943 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:50.070831    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f0195818-f111-47ca-a386-005ede718851-istiod-ca-cert\") pod \"e2e-shared-a-w2-2a25a4-openshift-default-7bfc77bdbf-9prd7\" (UID: \"f0195818-f111-47ca-a386-005ede718851\") " pod="openshift-ingress/e2e-shared-a-w2-2a25a4-openshift-default-7bfc77bdbf-9prd7"
Oct 30 22:08:50.078943 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:50.071880    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f0195818-f111-47ca-a386-005ede718851-istiod-ca-cert\") pod \"e2e-shared-a-w2-2a25a4-openshift-default-7bfc77bdbf-9prd7\" (UID: \"f0195818-f111-47ca-a386-005ede718851\") " pod="openshift-ingress/e2e-shared-a-w2-2a25a4-openshift-default-7bfc77bdbf-9prd7"
Oct 30 22:08:50.078943 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:50.072572    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0195818-f111-47ca-a386-005ede718851-workload-socket\") pod \"e2e-shared-a-w2-2a25a4-openshift-default-7bfc77bdbf-9prd7\" (UID: \"f0195818-f111-47ca-a386-005ede718851\") " pod="openshift-ingress/e2e-shared-a-w2-2a25a4-openshift-default-7bfc77bdbf-9prd7"
Oct 30 22:08:50.090958 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:50.080144    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0195818-f111-47ca-a386-005ede718851-credential-socket\") pod \"e2e-shared-a-w2-2a25a4-openshift-default-7bfc77bdbf-9prd7\" (UID: \"f0195818-f111-47ca-a386-005ede718851\") " pod="openshift-ingress/e2e-shared-a-w2-2a25a4-openshift-default-7bfc77bdbf-9prd7"
Oct 30 22:08:50.090958 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:50.089066    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f0195818-f111-47ca-a386-005ede718851-istio-data\") pod \"e2e-shared-a-w2-2a25a4-openshift-default-7bfc77bdbf-9prd7\" (UID: \"f0195818-f111-47ca-a386-005ede718851\") " pod="openshift-ingress/e2e-shared-a-w2-2a25a4-openshift-default-7bfc77bdbf-9prd7"
Oct 30 22:08:50.149051 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:50.149019    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f0195818-f111-47ca-a386-005ede718851-workload-certs\") pod \"e2e-shared-a-w2-2a25a4-openshift-default-7bfc77bdbf-9prd7\" (UID: \"f0195818-f111-47ca-a386-005ede718851\") " pod="openshift-ingress/e2e-shared-a-w2-2a25a4-openshift-default-7bfc77bdbf-9prd7"
Oct 30 22:08:50.192609 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:50.192537    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f0195818-f111-47ca-a386-005ede718851-istio-envoy\") pod \"e2e-shared-a-w2-2a25a4-openshift-default-7bfc77bdbf-9prd7\" (UID: \"f0195818-f111-47ca-a386-005ede718851\") " pod="openshift-ingress/e2e-shared-a-w2-2a25a4-openshift-default-7bfc77bdbf-9prd7"
Oct 30 22:08:50.233053 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:50.196368    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f0195818-f111-47ca-a386-005ede718851-istio-podinfo\") pod \"e2e-shared-a-w2-2a25a4-openshift-default-7bfc77bdbf-9prd7\" (UID: \"f0195818-f111-47ca-a386-005ede718851\") " pod="openshift-ingress/e2e-shared-a-w2-2a25a4-openshift-default-7bfc77bdbf-9prd7"
Oct 30 22:08:50.233053 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:50.204888    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f0195818-f111-47ca-a386-005ede718851-istio-token\") pod \"e2e-shared-a-w2-2a25a4-openshift-default-7bfc77bdbf-9prd7\" (UID: \"f0195818-f111-47ca-a386-005ede718851\") " pod="openshift-ingress/e2e-shared-a-w2-2a25a4-openshift-default-7bfc77bdbf-9prd7"
Oct 30 22:08:50.233053 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:50.205526    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xlzr6\" (UniqueName: \"kubernetes.io/projected/f0195818-f111-47ca-a386-005ede718851-kube-api-access-xlzr6\") pod \"e2e-shared-a-w2-2a25a4-openshift-default-7bfc77bdbf-9prd7\" (UID: \"f0195818-f111-47ca-a386-005ede718851\") " pod="openshift-ingress/e2e-shared-a-w2-2a25a4-openshift-default-7bfc77bdbf-9prd7"
Oct 30 22:08:50.339943 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:50.331666    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-2a25a4-openshift-default-7bfc77bdbf-9prd7"
Oct 30 22:08:50.542944 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:08:50.508417    4473 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fncvg" containerName="extractor"
Oct 30 22:08:50.946485 ip-10-0-140-18 kubenswrapper[4473]: W1030 22:08:50.946451    4473 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0195818_f111_47ca_a386_005ede718851.slice/crio-08e3a27608aea71044a60e510966e054159261031c7896f8a552cb01906f06ca WatchSource:0}: Error finding container 08e3a27608aea71044a60e510966e054159261031c7896f8a552cb01906f06ca: Status 404 returned error can't find the container with id 08e3a27608aea71044a60e510966e054159261031c7896f8a552cb01906f06ca
Oct 30 22:08:50.948575 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:50.948551    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-2a25a4-openshift-default-7bfc77bdbf-9prd7"]
Oct 30 22:08:50.949688 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:50.949654    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:08:50.949766 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:50.949717    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:08:50.949766 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:50.949745    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:08:51.665174 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:51.665137    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-2a25a4-openshift-default-7bfc77bdbf-9prd7" event={"ID":"f0195818-f111-47ca-a386-005ede718851","Type":"ContainerStarted","Data":"4d00e9f44c1f24b9b0a4c0d1666f42731d1a25290cdb7c1424a51317a0f45cc2"}
Oct 30 22:08:51.665359 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:51.665191    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-2a25a4-openshift-default-7bfc77bdbf-9prd7" event={"ID":"f0195818-f111-47ca-a386-005ede718851","Type":"ContainerStarted","Data":"08e3a27608aea71044a60e510966e054159261031c7896f8a552cb01906f06ca"}
Oct 30 22:08:51.793443 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:51.793391    4473 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w2-2a25a4-openshift-default-7bfc77bdbf-9prd7" podStartSLOduration=2.79337421 podStartE2EDuration="2.79337421s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:08:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:08:51.790038155 +0000 UTC m=+1577.563406646" watchObservedRunningTime="2026-10-30 22:08:51.79337421 +0000 UTC m=+1577.566742696"
Oct 30 22:08:51.845170 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:51.845135    4473 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-90f9e9d3-64b667bb5-hjf95"]
Oct 30 22:08:51.846596 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:51.846555    4473 cpu_manager.go:414] "containerMap: removing container" podUID="e85a3a56-ad99-4c7f-bfbf-c858e78f1dd0" containerName="istio-proxy"
Oct 30 22:08:51.846596 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:51.846578    4473 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e85a3a56-ad99-4c7f-bfbf-c858e78f1dd0" containerName="istio-proxy"
Oct 30 22:08:51.851279 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:51.851258    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-90f9e9d3-64b667bb5-hjf95"
Oct 30 22:08:51.934027 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:51.933076    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-90f9e9d3-serving-cert\""
Oct 30 22:08:51.941969 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:51.940016    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1e012266-60e2-4497-b8b1-9524e23c3421-openshift-service-ca\") pod \"maas-api-e2e-migrate-90f9e9d3-64b667bb5-hjf95\" (UID: \"1e012266-60e2-4497-b8b1-9524e23c3421\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-90f9e9d3-64b667bb5-hjf95"
Oct 30 22:08:51.941969 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:51.940084    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1e012266-60e2-4497-b8b1-9524e23c3421-maas-api-tls\") pod \"maas-api-e2e-migrate-90f9e9d3-64b667bb5-hjf95\" (UID: \"1e012266-60e2-4497-b8b1-9524e23c3421\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-90f9e9d3-64b667bb5-hjf95"
Oct 30 22:08:51.941969 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:51.940262    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hz5lk\" (UniqueName: \"kubernetes.io/projected/1e012266-60e2-4497-b8b1-9524e23c3421-kube-api-access-hz5lk\") pod \"maas-api-e2e-migrate-90f9e9d3-64b667bb5-hjf95\" (UID: \"1e012266-60e2-4497-b8b1-9524e23c3421\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-90f9e9d3-64b667bb5-hjf95"
Oct 30 22:08:51.941969 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:51.940345    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1e012266-60e2-4497-b8b1-9524e23c3421-metrics-certs\") pod \"maas-api-e2e-migrate-90f9e9d3-64b667bb5-hjf95\" (UID: \"1e012266-60e2-4497-b8b1-9524e23c3421\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-90f9e9d3-64b667bb5-hjf95"
Oct 30 22:08:51.970689 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:51.970644    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-90f9e9d3-64b667bb5-hjf95"]
Oct 30 22:08:52.108764 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:52.108721    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1e012266-60e2-4497-b8b1-9524e23c3421-openshift-service-ca\") pod \"maas-api-e2e-migrate-90f9e9d3-64b667bb5-hjf95\" (UID: \"1e012266-60e2-4497-b8b1-9524e23c3421\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-90f9e9d3-64b667bb5-hjf95"
Oct 30 22:08:52.126519 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:52.126482    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1e012266-60e2-4497-b8b1-9524e23c3421-maas-api-tls\") pod \"maas-api-e2e-migrate-90f9e9d3-64b667bb5-hjf95\" (UID: \"1e012266-60e2-4497-b8b1-9524e23c3421\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-90f9e9d3-64b667bb5-hjf95"
Oct 30 22:08:52.126908 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:52.126881    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hz5lk\" (UniqueName: \"kubernetes.io/projected/1e012266-60e2-4497-b8b1-9524e23c3421-kube-api-access-hz5lk\") pod \"maas-api-e2e-migrate-90f9e9d3-64b667bb5-hjf95\" (UID: \"1e012266-60e2-4497-b8b1-9524e23c3421\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-90f9e9d3-64b667bb5-hjf95"
Oct 30 22:08:52.127159 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:52.127139    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1e012266-60e2-4497-b8b1-9524e23c3421-metrics-certs\") pod \"maas-api-e2e-migrate-90f9e9d3-64b667bb5-hjf95\" (UID: \"1e012266-60e2-4497-b8b1-9524e23c3421\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-90f9e9d3-64b667bb5-hjf95"
Oct 30 22:08:52.128355 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:08:52.128294    4473 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-migrate-90f9e9d3-serving-cert: secret "maas-api-e2e-migrate-90f9e9d3-serving-cert" not found
Oct 30 22:08:52.129036 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:08:52.129018    4473 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1e012266-60e2-4497-b8b1-9524e23c3421-maas-api-tls podName:1e012266-60e2-4497-b8b1-9524e23c3421 nodeName:}" failed. No retries permitted until 2026-10-30 22:08:52.628984283 +0000 UTC m=+1578.402352750 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/1e012266-60e2-4497-b8b1-9524e23c3421-maas-api-tls") pod "maas-api-e2e-migrate-90f9e9d3-64b667bb5-hjf95" (UID: "1e012266-60e2-4497-b8b1-9524e23c3421") : secret "maas-api-e2e-migrate-90f9e9d3-serving-cert" not found
Oct 30 22:08:52.129384 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:52.111386    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1e012266-60e2-4497-b8b1-9524e23c3421-openshift-service-ca\") pod \"maas-api-e2e-migrate-90f9e9d3-64b667bb5-hjf95\" (UID: \"1e012266-60e2-4497-b8b1-9524e23c3421\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-90f9e9d3-64b667bb5-hjf95"
Oct 30 22:08:52.269020 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:52.268856    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hz5lk\" (UniqueName: \"kubernetes.io/projected/1e012266-60e2-4497-b8b1-9524e23c3421-kube-api-access-hz5lk\") pod \"maas-api-e2e-migrate-90f9e9d3-64b667bb5-hjf95\" (UID: \"1e012266-60e2-4497-b8b1-9524e23c3421\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-90f9e9d3-64b667bb5-hjf95"
Oct 30 22:08:52.277591 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:52.277556    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1e012266-60e2-4497-b8b1-9524e23c3421-metrics-certs\") pod \"maas-api-e2e-migrate-90f9e9d3-64b667bb5-hjf95\" (UID: \"1e012266-60e2-4497-b8b1-9524e23c3421\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-90f9e9d3-64b667bb5-hjf95"
Oct 30 22:08:52.344958 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:52.343339    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w2-2a25a4-openshift-default-7bfc77bdbf-9prd7"
Oct 30 22:08:52.416376 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:52.416346    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w2-2a25a4-openshift-default-7bfc77bdbf-9prd7"
Oct 30 22:08:52.651070 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:52.651038    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1e012266-60e2-4497-b8b1-9524e23c3421-maas-api-tls\") pod \"maas-api-e2e-migrate-90f9e9d3-64b667bb5-hjf95\" (UID: \"1e012266-60e2-4497-b8b1-9524e23c3421\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-90f9e9d3-64b667bb5-hjf95"
Oct 30 22:08:52.653790 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:52.653760    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1e012266-60e2-4497-b8b1-9524e23c3421-maas-api-tls\") pod \"maas-api-e2e-migrate-90f9e9d3-64b667bb5-hjf95\" (UID: \"1e012266-60e2-4497-b8b1-9524e23c3421\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-90f9e9d3-64b667bb5-hjf95"
Oct 30 22:08:52.672399 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:52.672367    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w2-2a25a4-openshift-default-7bfc77bdbf-9prd7"
Oct 30 22:08:52.673560 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:52.673540    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w2-2a25a4-openshift-default-7bfc77bdbf-9prd7"
Oct 30 22:08:52.773296 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:52.773258    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-90f9e9d3-64b667bb5-hjf95"
Oct 30 22:08:53.190447 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:53.190415    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-90f9e9d3-64b667bb5-hjf95"]
Oct 30 22:08:53.191203 ip-10-0-140-18 kubenswrapper[4473]: W1030 22:08:53.191173    4473 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e012266_60e2_4497_b8b1_9524e23c3421.slice/crio-83e2b0888d7435da9a060576ed44f302917fecac23e94efee670525953466a8f WatchSource:0}: Error finding container 83e2b0888d7435da9a060576ed44f302917fecac23e94efee670525953466a8f: Status 404 returned error can't find the container with id 83e2b0888d7435da9a060576ed44f302917fecac23e94efee670525953466a8f
Oct 30 22:08:53.680469 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:53.680430    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-90f9e9d3-64b667bb5-hjf95" event={"ID":"1e012266-60e2-4497-b8b1-9524e23c3421","Type":"ContainerStarted","Data":"83e2b0888d7435da9a060576ed44f302917fecac23e94efee670525953466a8f"}
Oct 30 22:08:54.689182 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:54.689142    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-90f9e9d3-64b667bb5-hjf95" event={"ID":"1e012266-60e2-4497-b8b1-9524e23c3421","Type":"ContainerStarted","Data":"721f97e99ef368ce4eff984bbce04d94dbf8d541a2b655ddc3f8b5bdcecbf1a7"}
Oct 30 22:08:54.689687 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:54.689245    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-90f9e9d3-64b667bb5-hjf95"
Oct 30 22:08:54.859291 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:54.859244    4473 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-90f9e9d3-64b667bb5-hjf95" podStartSLOduration=3.054993432 podStartE2EDuration="3.859228133s" totalImagesPullingTime="804.234701ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:08:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:08:54.856363541 +0000 UTC m=+1580.629732029" watchObservedRunningTime="2026-10-30 22:08:54.859228133 +0000 UTC m=+1580.632596623"
Oct 30 22:08:59.191945 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:59.156891    4473 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-2a25a4-75b5bd6c5-8fcr9"]
Oct 30 22:08:59.191945 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:59.167514    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-2a25a4-75b5bd6c5-8fcr9"
Oct 30 22:08:59.255073 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:59.250607    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-2a25a4-75b5bd6c5-8fcr9"]
Oct 30 22:08:59.293420 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:59.293387    4473 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w2-2a25a4\""
Oct 30 22:08:59.293814 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:59.293788    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w2-2a25a4-dockercfg-sdwxh\""
Oct 30 22:08:59.312713 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:59.298544    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/050a6565-0515-4f27-8110-51262e515fa2-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-2a25a4-75b5bd6c5-8fcr9\" (UID: \"050a6565-0515-4f27-8110-51262e515fa2\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-2a25a4-75b5bd6c5-8fcr9"
Oct 30 22:08:59.312713 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:59.298690    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hchcz\" (UniqueName: \"kubernetes.io/projected/050a6565-0515-4f27-8110-51262e515fa2-kube-api-access-hchcz\") pod \"payload-processing-e2e-shared-a-w2-2a25a4-75b5bd6c5-8fcr9\" (UID: \"050a6565-0515-4f27-8110-51262e515fa2\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-2a25a4-75b5bd6c5-8fcr9"
Oct 30 22:08:59.400733 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:59.400693    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/050a6565-0515-4f27-8110-51262e515fa2-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-2a25a4-75b5bd6c5-8fcr9\" (UID: \"050a6565-0515-4f27-8110-51262e515fa2\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-2a25a4-75b5bd6c5-8fcr9"
Oct 30 22:08:59.400945 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:59.400827    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hchcz\" (UniqueName: \"kubernetes.io/projected/050a6565-0515-4f27-8110-51262e515fa2-kube-api-access-hchcz\") pod \"payload-processing-e2e-shared-a-w2-2a25a4-75b5bd6c5-8fcr9\" (UID: \"050a6565-0515-4f27-8110-51262e515fa2\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-2a25a4-75b5bd6c5-8fcr9"
Oct 30 22:08:59.402730 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:59.402512    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/050a6565-0515-4f27-8110-51262e515fa2-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-2a25a4-75b5bd6c5-8fcr9\" (UID: \"050a6565-0515-4f27-8110-51262e515fa2\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-2a25a4-75b5bd6c5-8fcr9"
Oct 30 22:08:59.469293 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:59.469216    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hchcz\" (UniqueName: \"kubernetes.io/projected/050a6565-0515-4f27-8110-51262e515fa2-kube-api-access-hchcz\") pod \"payload-processing-e2e-shared-a-w2-2a25a4-75b5bd6c5-8fcr9\" (UID: \"050a6565-0515-4f27-8110-51262e515fa2\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-2a25a4-75b5bd6c5-8fcr9"
Oct 30 22:08:59.629227 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:08:59.629184    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-2a25a4-75b5bd6c5-8fcr9"
Oct 30 22:09:00.018702 ip-10-0-140-18 kubenswrapper[4473]: W1030 22:09:00.018673    4473 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod050a6565_0515_4f27_8110_51262e515fa2.slice/crio-6a980c7da3b5a91a7df25f9498d3a48f908e066c0abf00e6d4da124a1f8fba88 WatchSource:0}: Error finding container 6a980c7da3b5a91a7df25f9498d3a48f908e066c0abf00e6d4da124a1f8fba88: Status 404 returned error can't find the container with id 6a980c7da3b5a91a7df25f9498d3a48f908e066c0abf00e6d4da124a1f8fba88
Oct 30 22:09:00.020522 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:09:00.020498    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-2a25a4-75b5bd6c5-8fcr9"]
Oct 30 22:09:00.754329 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:09:00.754299    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-90f9e9d3-64b667bb5-hjf95"
Oct 30 22:09:00.806738 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:09:00.806700    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-2a25a4-75b5bd6c5-8fcr9" event={"ID":"050a6565-0515-4f27-8110-51262e515fa2","Type":"ContainerStarted","Data":"44f32e02c30bb001e535b9e164041d3f08d03267c615524eb11b06c187bfaca1"}
Oct 30 22:09:00.806934 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:09:00.806755    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-2a25a4-75b5bd6c5-8fcr9" event={"ID":"050a6565-0515-4f27-8110-51262e515fa2","Type":"ContainerStarted","Data":"6a980c7da3b5a91a7df25f9498d3a48f908e066c0abf00e6d4da124a1f8fba88"}
Oct 30 22:09:00.806934 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:09:00.806848    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-2a25a4-75b5bd6c5-8fcr9"
Oct 30 22:09:01.058858 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:09:01.058741    4473 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-2a25a4-75b5bd6c5-8fcr9" podStartSLOduration=3.058724796 podStartE2EDuration="3.058724796s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:08:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:09:01.034590537 +0000 UTC m=+1586.807959024" watchObservedRunningTime="2026-10-30 22:09:01.058724796 +0000 UTC m=+1586.832093276"
Oct 30 22:09:11.818836 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:09:11.818802    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-2a25a4-75b5bd6c5-8fcr9"
Oct 30 22:09:20.328597 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:09:20.328549    4473 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fncvg" containerName="extractor"
Oct 30 22:09:40.666912 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:09:40.666879    4473 generic.go:356] "Generic (PLEG): container finished" podID="bad44c70-cd8b-415c-8d9a-86515b14a266" containerID="67155ca335eefca79662fdd7a204683fc189865ba832b1fbc7ad95336c047da9" exitCode=0
Oct 30 22:09:40.667438 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:09:40.666960    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-90f9e9d3-gw-openshift-default-55569c489-fllqv" event={"ID":"bad44c70-cd8b-415c-8d9a-86515b14a266","Type":"ContainerDied","Data":"67155ca335eefca79662fdd7a204683fc189865ba832b1fbc7ad95336c047da9"}
Oct 30 22:09:40.667522 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:09:40.667495    4473 scope.go:122] "RemoveContainer" containerID="67155ca335eefca79662fdd7a204683fc189865ba832b1fbc7ad95336c047da9"
Oct 30 22:09:40.669270 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:09:40.669251    4473 generic.go:356] "Generic (PLEG): container finished" podID="233e0e6f-3447-4d9c-a443-0fa29a3b4adc" containerID="d155d219b8d5da74829b43cbb01d3e196699214e3ff6d95528e36803559bf89e" exitCode=0
Oct 30 22:09:40.669402 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:09:40.669308    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-497924bf-openshift-default-85499cb6f-fw82l" event={"ID":"233e0e6f-3447-4d9c-a443-0fa29a3b4adc","Type":"ContainerDied","Data":"d155d219b8d5da74829b43cbb01d3e196699214e3ff6d95528e36803559bf89e"}
Oct 30 22:09:40.669464 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:09:40.669416    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:09:40.669523 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:09:40.669509    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:09:40.669595 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:09:40.669564    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:09:40.669699 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:09:40.669684    4473 scope.go:122] "RemoveContainer" containerID="d155d219b8d5da74829b43cbb01d3e196699214e3ff6d95528e36803559bf89e"
Oct 30 22:09:40.671413 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:09:40.671385    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:09:40.671501 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:09:40.671485    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:09:40.671550 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:09:40.671535    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:09:41.686318 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:09:41.686285    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-497924bf-openshift-default-85499cb6f-fw82l" event={"ID":"233e0e6f-3447-4d9c-a443-0fa29a3b4adc","Type":"ContainerStarted","Data":"6216d1dac58cffb05954f1c0d99be4b05accde204017f6172781c223da299951"}
Oct 30 22:09:41.689082 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:09:41.689058    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-90f9e9d3-gw-openshift-default-55569c489-fllqv" event={"ID":"bad44c70-cd8b-415c-8d9a-86515b14a266","Type":"ContainerStarted","Data":"4d7037d25f80726307f2ea28f90ab71e550e3c346bb8ecdf92ca74a9ca9f2c7a"}
Oct 30 22:09:41.861949 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:09:41.861269    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-497924bf-openshift-default-85499cb6f-fw82l"
Oct 30 22:09:41.861949 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:09:41.861415    4473 patch_prober.go:29] interesting pod/e2e-derive-497924bf-openshift-default-85499cb6f-fw82l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.104:15021/healthz/ready\": dial tcp 10.133.0.104:15021: connect: connection refused" start-of-body=
Oct 30 22:09:41.861949 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:09:41.861456    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-497924bf-openshift-default-85499cb6f-fw82l" podUID="233e0e6f-3447-4d9c-a443-0fa29a3b4adc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.104:15021/healthz/ready\": dial tcp 10.133.0.104:15021: connect: connection refused"
Oct 30 22:09:42.456644 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:09:42.456609    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-90f9e9d3-gw-openshift-default-55569c489-fllqv"
Oct 30 22:09:42.458367 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:09:42.458339    4473 patch_prober.go:29] interesting pod/e2e-migrate-90f9e9d3-gw-openshift-default-55569c489-fllqv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.101:15021/healthz/ready\": dial tcp 10.133.0.101:15021: connect: connection refused" start-of-body=
Oct 30 22:09:42.458586 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:09:42.458560    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-90f9e9d3-gw-openshift-default-55569c489-fllqv" podUID="bad44c70-cd8b-415c-8d9a-86515b14a266" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.101:15021/healthz/ready\": dial tcp 10.133.0.101:15021: connect: connection refused"
Oct 30 22:09:42.867946 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:09:42.864610    4473 patch_prober.go:29] interesting pod/e2e-derive-497924bf-openshift-default-85499cb6f-fw82l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.104:15021/healthz/ready\": dial tcp 10.133.0.104:15021: connect: connection refused" start-of-body=
Oct 30 22:09:42.867946 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:09:42.864667    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-497924bf-openshift-default-85499cb6f-fw82l" podUID="233e0e6f-3447-4d9c-a443-0fa29a3b4adc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.104:15021/healthz/ready\": dial tcp 10.133.0.104:15021: connect: connection refused"
Oct 30 22:09:43.218772 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:09:43.218677    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-497924bf-openshift-default-85499cb6f-fw82l"
Oct 30 22:09:43.450383 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:09:43.450345    4473 patch_prober.go:29] interesting pod/e2e-migrate-90f9e9d3-gw-openshift-default-55569c489-fllqv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.101:15021/healthz/ready\": dial tcp 10.133.0.101:15021: connect: connection refused" start-of-body=
Oct 30 22:09:43.450647 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:09:43.450622    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-90f9e9d3-gw-openshift-default-55569c489-fllqv" podUID="bad44c70-cd8b-415c-8d9a-86515b14a266" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.101:15021/healthz/ready\": dial tcp 10.133.0.101:15021: connect: connection refused"
Oct 30 22:09:43.699231 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:09:43.699198    4473 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-242ddaf0-7b7b974bc-7q9bh"]
Oct 30 22:09:43.712796 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:09:43.712765    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-242ddaf0-7b7b974bc-7q9bh"
Oct 30 22:09:44.023013 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:09:43.923285    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/75c6c21f-c5c0-4f9e-9f12-621e01735dda-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-242ddaf0-7b7b974bc-7q9bh\" (UID: \"75c6c21f-c5c0-4f9e-9f12-621e01735dda\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-242ddaf0-7b7b974bc-7q9bh"
Oct 30 22:09:44.080970 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:09:44.080906    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tdvb5\" (UniqueName: \"kubernetes.io/projected/75c6c21f-c5c0-4f9e-9f12-621e01735dda-kube-api-access-tdvb5\") pod \"payload-pre-processing-e2e-mt-242ddaf0-7b7b974bc-7q9bh\" (UID: \"75c6c21f-c5c0-4f9e-9f12-621e01735dda\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-242ddaf0-7b7b974bc-7q9bh"
Oct 30 22:09:44.081408 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:09:43.949493    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-242ddaf0-7b7b974bc-7q9bh"]
Oct 30 22:09:44.128963 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:09:44.109136    4473 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-242ddaf0\""
Oct 30 22:09:44.128963 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:09:44.109771    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-242ddaf0-dockercfg-hr5sh\""
Oct 30 22:09:44.128963 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:09:44.115113    4473 patch_prober.go:29] interesting pod/e2e-derive-497924bf-openshift-default-85499cb6f-fw82l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.104:15021/healthz/ready\": dial tcp 10.133.0.104:15021: connect: connection refused" start-of-body=
Oct 30 22:09:44.128963 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:09:44.115159    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-497924bf-openshift-default-85499cb6f-fw82l" podUID="233e0e6f-3447-4d9c-a443-0fa29a3b4adc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.104:15021/healthz/ready\": dial tcp 10.133.0.104:15021: connect: connection refused"
Oct 30 22:09:44.243947 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:09:44.211569    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/75c6c21f-c5c0-4f9e-9f12-621e01735dda-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-242ddaf0-7b7b974bc-7q9bh\" (UID: \"75c6c21f-c5c0-4f9e-9f12-621e01735dda\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-242ddaf0-7b7b974bc-7q9bh"
Oct 30 22:09:44.243947 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:09:44.217398    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/75c6c21f-c5c0-4f9e-9f12-621e01735dda-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-242ddaf0-7b7b974bc-7q9bh\" (UID: \"75c6c21f-c5c0-4f9e-9f12-621e01735dda\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-242ddaf0-7b7b974bc-7q9bh"
Oct 30 22:09:44.346997 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:09:44.249761    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tdvb5\" (UniqueName: \"kubernetes.io/projected/75c6c21f-c5c0-4f9e-9f12-621e01735dda-kube-api-access-tdvb5\") pod \"payload-pre-processing-e2e-mt-242ddaf0-7b7b974bc-7q9bh\" (UID: \"75c6c21f-c5c0-4f9e-9f12-621e01735dda\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-242ddaf0-7b7b974bc-7q9bh"
Oct 30 22:09:44.432659 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:09:44.370961    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tdvb5\" (UniqueName: \"kubernetes.io/projected/75c6c21f-c5c0-4f9e-9f12-621e01735dda-kube-api-access-tdvb5\") pod \"payload-pre-processing-e2e-mt-242ddaf0-7b7b974bc-7q9bh\" (UID: \"75c6c21f-c5c0-4f9e-9f12-621e01735dda\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-242ddaf0-7b7b974bc-7q9bh"
Oct 30 22:09:44.432659 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:09:44.432285    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-242ddaf0-7b7b974bc-7q9bh"
Oct 30 22:09:44.630168 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:09:44.630135    4473 patch_prober.go:29] interesting pod/e2e-migrate-90f9e9d3-gw-openshift-default-55569c489-fllqv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.101:15021/healthz/ready\": dial tcp 10.133.0.101:15021: connect: connection refused" start-of-body=
Oct 30 22:09:44.630389 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:09:44.630361    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-90f9e9d3-gw-openshift-default-55569c489-fllqv" podUID="bad44c70-cd8b-415c-8d9a-86515b14a266" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.101:15021/healthz/ready\": dial tcp 10.133.0.101:15021: connect: connection refused"
Oct 30 22:09:44.839499 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:09:44.839460    4473 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-242ddaf0-576ccc4d89-m89lr"]
Oct 30 22:09:44.980989 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:09:44.921520    4473 patch_prober.go:29] interesting pod/e2e-derive-497924bf-openshift-default-85499cb6f-fw82l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.104:15021/healthz/ready\": dial tcp 10.133.0.104:15021: connect: connection refused" start-of-body=
Oct 30 22:09:44.981281 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:09:44.981228    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-497924bf-openshift-default-85499cb6f-fw82l" podUID="233e0e6f-3447-4d9c-a443-0fa29a3b4adc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.104:15021/healthz/ready\": dial tcp 10.133.0.104:15021: connect: connection refused"
Oct 30 22:09:44.998181 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:09:44.998136    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-242ddaf0-576ccc4d89-m89lr"]
Oct 30 22:09:44.998375 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:09:44.998197    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-bc2x8"]
Oct 30 22:09:44.998435 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:09:44.998409    4473 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-bc2x8" podUID="365aaba7-1b9d-463f-9231-b0edad7e1aff" containerName="istio-proxy" containerID="cri-o://c1fa63726fee81fdaba4280e965b3a92b716cdccf4e7edacc7bdf749e71e2546" gracePeriod=30
Oct 30 22:09:44.999029 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:09:44.998996    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-242ddaf0-576ccc4d89-m89lr"
Oct 30 22:09:45.000026 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:09:44.999843    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-pxzmr"]
Oct 30 22:09:45.000410 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:09:45.000381    4473 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-pxzmr" podUID="3552416b-6da1-4c28-be4b-5450abc40f79" containerName="istio-proxy" containerID="cri-o://42de881e49dd00d4bb57e44377c0ceda3c35c6e6b3cf143c396a68a2d0b4b5ec" gracePeriod=30
Oct 30 22:09:45.124447 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:09:45.124284    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/13ab2734-6186-480c-ae76-6f00a295e1e1-plugins-config-volume\") pod \"payload-processing-e2e-mt-242ddaf0-576ccc4d89-m89lr\" (UID: \"13ab2734-6186-480c-ae76-6f00a295e1e1\") " pod="openshift-ingress/payload-processing-e2e-mt-242ddaf0-576ccc4d89-m89lr"
Oct 30 22:09:45.124447 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:09:45.124335    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dqqsd\" (UniqueName: \"kubernetes.io/projected/13ab2734-6186-480c-ae76-6f00a295e1e1-kube-api-access-dqqsd\") pod \"payload-processing-e2e-mt-242ddaf0-576ccc4d89-m89lr\" (UID: \"13ab2734-6186-480c-ae76-6f00a295e1e1\") " pod="openshift-ingress/payload-processing-e2e-mt-242ddaf0-576ccc4d89-m89lr"
Oct 30 22:09:45.275030 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:09:45.271020    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/13ab2734-6186-480c-ae76-6f00a295e1e1-plugins-config-volume\") pod \"payload-processing-e2e-mt-242ddaf0-576ccc4d89-m89lr\" (UID: \"13ab2734-6186-480c-ae76-6f00a295e1e1\") " pod="openshift-ingress/payload-processing-e2e-mt-242ddaf0-576ccc4d89-m89lr"
Oct 30 22:09:45.275030 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:09:45.271072    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dqqsd\" (UniqueName: \"kubernetes.io/projected/13ab2734-6186-480c-ae76-6f00a295e1e1-kube-api-access-dqqsd\") pod \"payload-processing-e2e-mt-242ddaf0-576ccc4d89-m89lr\" (UID: \"13ab2734-6186-480c-ae76-6f00a295e1e1\") " pod="openshift-ingress/payload-processing-e2e-mt-242ddaf0-576ccc4d89-m89lr"
Oct 30 22:09:45.285225 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:09:45.285188    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/13ab2734-6186-480c-ae76-6f00a295e1e1-plugins-config-volume\") pod \"payload-processing-e2e-mt-242ddaf0-576ccc4d89-m89lr\" (UID: \"13ab2734-6186-480c-ae76-6f00a295e1e1\") " pod="openshift-ingress/payload-processing-e2e-mt-242ddaf0-576ccc4d89-m89lr"
Oct 30 22:09:45.385266 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:09:45.384024    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dqqsd\" (UniqueName: \"kubernetes.io/projected/13ab2734-6186-480c-ae76-6f00a295e1e1-kube-api-access-dqqsd\") pod \"payload-processing-e2e-mt-242ddaf0-576ccc4d89-m89lr\" (UID: \"13ab2734-6186-480c-ae76-6f00a295e1e1\") " pod="openshift-ingress/payload-processing-e2e-mt-242ddaf0-576ccc4d89-m89lr"
Oct 30 22:09:45.386510 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:09:45.386481    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-242ddaf0-7b7b974bc-7q9bh"]
Oct 30 22:09:45.388510 ip-10-0-140-18 kubenswrapper[4473]: W1030 22:09:45.388482    4473 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod75c6c21f_c5c0_4f9e_9f12_621e01735dda.slice/crio-b7f391698912f145de2270d9c59a07635acb309bdf05845667ae8c5b1a8855cc WatchSource:0}: Error finding container b7f391698912f145de2270d9c59a07635acb309bdf05845667ae8c5b1a8855cc: Status 404 returned error can't find the container with id b7f391698912f145de2270d9c59a07635acb309bdf05845667ae8c5b1a8855cc
Oct 30 22:09:45.457948 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:09:45.455413    4473 patch_prober.go:29] interesting pod/e2e-migrate-90f9e9d3-gw-openshift-default-55569c489-fllqv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.101:15021/healthz/ready\": dial tcp 10.133.0.101:15021: connect: connection refused" start-of-body=
Oct 30 22:09:45.457948 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:09:45.455473    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-90f9e9d3-gw-openshift-default-55569c489-fllqv" podUID="bad44c70-cd8b-415c-8d9a-86515b14a266" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.101:15021/healthz/ready\": dial tcp 10.133.0.101:15021: connect: connection refused"
Oct 30 22:09:45.655292 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:09:45.655255    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-242ddaf0-576ccc4d89-m89lr"
Oct 30 22:09:45.861961 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:09:45.857594    4473 patch_prober.go:29] interesting pod/e2e-derive-497924bf-openshift-default-85499cb6f-fw82l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.104:15021/healthz/ready\": dial tcp 10.133.0.104:15021: connect: connection refused" start-of-body=
Oct 30 22:09:45.861961 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:09:45.857654    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-497924bf-openshift-default-85499cb6f-fw82l" podUID="233e0e6f-3447-4d9c-a443-0fa29a3b4adc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.104:15021/healthz/ready\": dial tcp 10.133.0.104:15021: connect: connection refused"
Oct 30 22:09:46.023154 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:09:46.023119    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-242ddaf0-7b7b974bc-7q9bh" event={"ID":"75c6c21f-c5c0-4f9e-9f12-621e01735dda","Type":"ContainerStarted","Data":"d7ee2d15da605bc96fb1440f998fef85d432313e82010bee47ac4cdcf870ffbf"}
Oct 30 22:09:46.023392 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:09:46.023181    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-242ddaf0-7b7b974bc-7q9bh" event={"ID":"75c6c21f-c5c0-4f9e-9f12-621e01735dda","Type":"ContainerStarted","Data":"b7f391698912f145de2270d9c59a07635acb309bdf05845667ae8c5b1a8855cc"}
Oct 30 22:09:46.023495 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:09:46.023458    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-242ddaf0-7b7b974bc-7q9bh"
Oct 30 22:09:46.164515 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:09:46.164444    4473 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-242ddaf0-7b7b974bc-7q9bh" podStartSLOduration=3.16442106 podStartE2EDuration="3.16442106s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:09:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:09:46.162632912 +0000 UTC m=+1631.936001402" watchObservedRunningTime="2026-10-30 22:09:46.16442106 +0000 UTC m=+1631.937789548"
Oct 30 22:09:46.208559 ip-10-0-140-18 kubenswrapper[4473]: W1030 22:09:46.208524    4473 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod13ab2734_6186_480c_ae76_6f00a295e1e1.slice/crio-8150cba6f4275de9af661856b9917909059d38735630f363d2e7fa83a42abf8a WatchSource:0}: Error finding container 8150cba6f4275de9af661856b9917909059d38735630f363d2e7fa83a42abf8a: Status 404 returned error can't find the container with id 8150cba6f4275de9af661856b9917909059d38735630f363d2e7fa83a42abf8a
Oct 30 22:09:46.208787 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:09:46.208763    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-242ddaf0-576ccc4d89-m89lr"]
Oct 30 22:09:46.467018 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:09:46.457308    4473 patch_prober.go:29] interesting pod/e2e-migrate-90f9e9d3-gw-openshift-default-55569c489-fllqv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.101:15021/healthz/ready\": dial tcp 10.133.0.101:15021: connect: connection refused" start-of-body=
Oct 30 22:09:46.467018 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:09:46.457383    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-90f9e9d3-gw-openshift-default-55569c489-fllqv" podUID="bad44c70-cd8b-415c-8d9a-86515b14a266" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.101:15021/healthz/ready\": dial tcp 10.133.0.101:15021: connect: connection refused"
Oct 30 22:09:46.863473 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:09:46.863441    4473 patch_prober.go:29] interesting pod/e2e-derive-497924bf-openshift-default-85499cb6f-fw82l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.104:15021/healthz/ready\": dial tcp 10.133.0.104:15021: connect: connection refused" start-of-body=
Oct 30 22:09:46.863749 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:09:46.863709    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-497924bf-openshift-default-85499cb6f-fw82l" podUID="233e0e6f-3447-4d9c-a443-0fa29a3b4adc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.104:15021/healthz/ready\": dial tcp 10.133.0.104:15021: connect: connection refused"
Oct 30 22:09:47.099450 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:09:47.099384    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-242ddaf0-576ccc4d89-m89lr" event={"ID":"13ab2734-6186-480c-ae76-6f00a295e1e1","Type":"ContainerStarted","Data":"dfac5ec330b025a5580505fb7749c765597cd13f18c3ffb06148e38e54655d7a"}
Oct 30 22:09:47.099668 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:09:47.099495    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-242ddaf0-576ccc4d89-m89lr" event={"ID":"13ab2734-6186-480c-ae76-6f00a295e1e1","Type":"ContainerStarted","Data":"8150cba6f4275de9af661856b9917909059d38735630f363d2e7fa83a42abf8a"}
Oct 30 22:09:47.099822 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:09:47.099795    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-242ddaf0-576ccc4d89-m89lr"
Oct 30 22:09:47.220165 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:09:47.220046    4473 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-242ddaf0-576ccc4d89-m89lr" podStartSLOduration=4.220033053 podStartE2EDuration="4.220033053s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:09:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:09:47.219812792 +0000 UTC m=+1632.993181280" watchObservedRunningTime="2026-10-30 22:09:47.220033053 +0000 UTC m=+1632.993401539"
Oct 30 22:09:47.456948 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:09:47.455370    4473 patch_prober.go:29] interesting pod/e2e-migrate-90f9e9d3-gw-openshift-default-55569c489-fllqv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.101:15021/healthz/ready\": dial tcp 10.133.0.101:15021: connect: connection refused" start-of-body=
Oct 30 22:09:47.456948 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:09:47.455434    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-90f9e9d3-gw-openshift-default-55569c489-fllqv" podUID="bad44c70-cd8b-415c-8d9a-86515b14a266" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.101:15021/healthz/ready\": dial tcp 10.133.0.101:15021: connect: connection refused"
Oct 30 22:09:47.872673 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:09:47.872550    4473 patch_prober.go:29] interesting pod/e2e-derive-497924bf-openshift-default-85499cb6f-fw82l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.104:15021/healthz/ready\": dial tcp 10.133.0.104:15021: connect: connection refused" start-of-body=
Oct 30 22:09:47.872673 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:09:47.872620    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-497924bf-openshift-default-85499cb6f-fw82l" podUID="233e0e6f-3447-4d9c-a443-0fa29a3b4adc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.104:15021/healthz/ready\": dial tcp 10.133.0.104:15021: connect: connection refused"
Oct 30 22:09:48.454588 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:09:48.454547    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-90f9e9d3-gw-openshift-default-55569c489-fllqv"
Oct 30 22:09:48.455025 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:09:48.454955    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-90f9e9d3-gw-openshift-default-55569c489-fllqv"
Oct 30 22:09:48.455809 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:09:48.455791    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-90f9e9d3-gw-openshift-default-55569c489-fllqv"
Oct 30 22:09:48.877807 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:09:48.877776    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-497924bf-openshift-default-85499cb6f-fw82l"
Oct 30 22:09:48.879101 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:09:48.879081    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-497924bf-openshift-default-85499cb6f-fw82l"
Oct 30 22:09:50.129878 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:09:50.129850    4473 generic.go:356] "Generic (PLEG): container finished" podID="3552416b-6da1-4c28-be4b-5450abc40f79" containerID="42de881e49dd00d4bb57e44377c0ceda3c35c6e6b3cf143c396a68a2d0b4b5ec" exitCode=0
Oct 30 22:09:50.130361 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:09:50.129974    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-pxzmr" event={"ID":"3552416b-6da1-4c28-be4b-5450abc40f79","Type":"ContainerDied","Data":"42de881e49dd00d4bb57e44377c0ceda3c35c6e6b3cf143c396a68a2d0b4b5ec"}
Oct 30 22:09:50.131775 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:09:50.131749    4473 generic.go:356] "Generic (PLEG): container finished" podID="365aaba7-1b9d-463f-9231-b0edad7e1aff" containerID="c1fa63726fee81fdaba4280e965b3a92b716cdccf4e7edacc7bdf749e71e2546" exitCode=0
Oct 30 22:09:50.131902 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:09:50.131825    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-bc2x8" event={"ID":"365aaba7-1b9d-463f-9231-b0edad7e1aff","Type":"ContainerDied","Data":"c1fa63726fee81fdaba4280e965b3a92b716cdccf4e7edacc7bdf749e71e2546"}
Oct 30 22:09:50.331072 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:09:50.330980    4473 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fncvg" containerName="extractor"
Oct 30 22:09:50.902904 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:09:50.902872    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-bc2x8"
Oct 30 22:09:50.951601 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:09:50.951574    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-pxzmr"
Oct 30 22:09:51.019253 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:09:51.019163    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/365aaba7-1b9d-463f-9231-b0edad7e1aff-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/365aaba7-1b9d-463f-9231-b0edad7e1aff-workload-socket\") pod \"365aaba7-1b9d-463f-9231-b0edad7e1aff\" (UID: \"365aaba7-1b9d-463f-9231-b0edad7e1aff\") "
Oct 30 22:09:51.019469 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:09:51.019302    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3552416b-6da1-4c28-be4b-5450abc40f79-istio-token\" (UniqueName: \"kubernetes.io/projected/3552416b-6da1-4c28-be4b-5450abc40f79-istio-token\") pod \"3552416b-6da1-4c28-be4b-5450abc40f79\" (UID: \"3552416b-6da1-4c28-be4b-5450abc40f79\") "
Oct 30 22:09:51.019469 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:09:51.019423    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/365aaba7-1b9d-463f-9231-b0edad7e1aff-istio-token\" (UniqueName: \"kubernetes.io/projected/365aaba7-1b9d-463f-9231-b0edad7e1aff-istio-token\") pod \"365aaba7-1b9d-463f-9231-b0edad7e1aff\" (UID: \"365aaba7-1b9d-463f-9231-b0edad7e1aff\") "
Oct 30 22:09:51.019469 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:09:51.019452    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3552416b-6da1-4c28-be4b-5450abc40f79-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3552416b-6da1-4c28-be4b-5450abc40f79-istiod-ca-cert\") pod \"3552416b-6da1-4c28-be4b-5450abc40f79\" (UID: \"3552416b-6da1-4c28-be4b-5450abc40f79\") "
Oct 30 22:09:51.019469 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:09:51.019452    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/365aaba7-1b9d-463f-9231-b0edad7e1aff-workload-socket" pod "365aaba7-1b9d-463f-9231-b0edad7e1aff" (UID: "365aaba7-1b9d-463f-9231-b0edad7e1aff"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:09:51.019696 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:09:51.019541    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/3552416b-6da1-4c28-be4b-5450abc40f79-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3552416b-6da1-4c28-be4b-5450abc40f79-istio-podinfo\") pod \"3552416b-6da1-4c28-be4b-5450abc40f79\" (UID: \"3552416b-6da1-4c28-be4b-5450abc40f79\") "
Oct 30 22:09:51.019696 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:09:51.019577    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3552416b-6da1-4c28-be4b-5450abc40f79-kube-api-access-n825w\" (UniqueName: \"kubernetes.io/projected/3552416b-6da1-4c28-be4b-5450abc40f79-kube-api-access-n825w\") pod \"3552416b-6da1-4c28-be4b-5450abc40f79\" (UID: \"3552416b-6da1-4c28-be4b-5450abc40f79\") "
Oct 30 22:09:51.019696 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:09:51.019658    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/365aaba7-1b9d-463f-9231-b0edad7e1aff-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/365aaba7-1b9d-463f-9231-b0edad7e1aff-istiod-ca-cert\") pod \"365aaba7-1b9d-463f-9231-b0edad7e1aff\" (UID: \"365aaba7-1b9d-463f-9231-b0edad7e1aff\") "
Oct 30 22:09:51.019696 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:09:51.019685    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3552416b-6da1-4c28-be4b-5450abc40f79-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3552416b-6da1-4c28-be4b-5450abc40f79-istio-data\") pod \"3552416b-6da1-4c28-be4b-5450abc40f79\" (UID: \"3552416b-6da1-4c28-be4b-5450abc40f79\") "
Oct 30 22:09:51.019906 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:09:51.019722    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/365aaba7-1b9d-463f-9231-b0edad7e1aff-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/365aaba7-1b9d-463f-9231-b0edad7e1aff-istio-envoy\") pod \"365aaba7-1b9d-463f-9231-b0edad7e1aff\" (UID: \"365aaba7-1b9d-463f-9231-b0edad7e1aff\") "
Oct 30 22:09:51.019906 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:09:51.019751    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3552416b-6da1-4c28-be4b-5450abc40f79-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3552416b-6da1-4c28-be4b-5450abc40f79-workload-socket\") pod \"3552416b-6da1-4c28-be4b-5450abc40f79\" (UID: \"3552416b-6da1-4c28-be4b-5450abc40f79\") "
Oct 30 22:09:51.019906 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:09:51.019819    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/365aaba7-1b9d-463f-9231-b0edad7e1aff-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/365aaba7-1b9d-463f-9231-b0edad7e1aff-credential-socket\") pod \"365aaba7-1b9d-463f-9231-b0edad7e1aff\" (UID: \"365aaba7-1b9d-463f-9231-b0edad7e1aff\") "
Oct 30 22:09:51.019906 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:09:51.019863    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3552416b-6da1-4c28-be4b-5450abc40f79-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3552416b-6da1-4c28-be4b-5450abc40f79-istio-envoy\") pod \"3552416b-6da1-4c28-be4b-5450abc40f79\" (UID: \"3552416b-6da1-4c28-be4b-5450abc40f79\") "
Oct 30 22:09:51.020132 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:09:51.019942    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/365aaba7-1b9d-463f-9231-b0edad7e1aff-kube-api-access-qg69x\" (UniqueName: \"kubernetes.io/projected/365aaba7-1b9d-463f-9231-b0edad7e1aff-kube-api-access-qg69x\") pod \"365aaba7-1b9d-463f-9231-b0edad7e1aff\" (UID: \"365aaba7-1b9d-463f-9231-b0edad7e1aff\") "
Oct 30 22:09:51.020132 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:09:51.019995    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3552416b-6da1-4c28-be4b-5450abc40f79-istiod-ca-cert" pod "3552416b-6da1-4c28-be4b-5450abc40f79" (UID: "3552416b-6da1-4c28-be4b-5450abc40f79"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:09:51.020132 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:09:51.020023    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3552416b-6da1-4c28-be4b-5450abc40f79-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3552416b-6da1-4c28-be4b-5450abc40f79-workload-certs\") pod \"3552416b-6da1-4c28-be4b-5450abc40f79\" (UID: \"3552416b-6da1-4c28-be4b-5450abc40f79\") "
Oct 30 22:09:51.020132 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:09:51.020059    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/365aaba7-1b9d-463f-9231-b0edad7e1aff-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/365aaba7-1b9d-463f-9231-b0edad7e1aff-istio-data\") pod \"365aaba7-1b9d-463f-9231-b0edad7e1aff\" (UID: \"365aaba7-1b9d-463f-9231-b0edad7e1aff\") "
Oct 30 22:09:51.020132 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:09:51.020110    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3552416b-6da1-4c28-be4b-5450abc40f79-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3552416b-6da1-4c28-be4b-5450abc40f79-credential-socket\") pod \"3552416b-6da1-4c28-be4b-5450abc40f79\" (UID: \"3552416b-6da1-4c28-be4b-5450abc40f79\") "
Oct 30 22:09:51.020132 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:09:51.020113    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3552416b-6da1-4c28-be4b-5450abc40f79-istio-data" pod "3552416b-6da1-4c28-be4b-5450abc40f79" (UID: "3552416b-6da1-4c28-be4b-5450abc40f79"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:09:51.020611 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:09:51.020153    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/365aaba7-1b9d-463f-9231-b0edad7e1aff-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/365aaba7-1b9d-463f-9231-b0edad7e1aff-workload-certs\") pod \"365aaba7-1b9d-463f-9231-b0edad7e1aff\" (UID: \"365aaba7-1b9d-463f-9231-b0edad7e1aff\") "
Oct 30 22:09:51.020611 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:09:51.020182    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/365aaba7-1b9d-463f-9231-b0edad7e1aff-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/365aaba7-1b9d-463f-9231-b0edad7e1aff-istio-podinfo\") pod \"365aaba7-1b9d-463f-9231-b0edad7e1aff\" (UID: \"365aaba7-1b9d-463f-9231-b0edad7e1aff\") "
Oct 30 22:09:51.020611 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:09:51.020386    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/365aaba7-1b9d-463f-9231-b0edad7e1aff-credential-socket" pod "365aaba7-1b9d-463f-9231-b0edad7e1aff" (UID: "365aaba7-1b9d-463f-9231-b0edad7e1aff"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:09:51.020611 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:09:51.020597    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/365aaba7-1b9d-463f-9231-b0edad7e1aff-istiod-ca-cert" pod "365aaba7-1b9d-463f-9231-b0edad7e1aff" (UID: "365aaba7-1b9d-463f-9231-b0edad7e1aff"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:09:51.021370 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:09:51.021340    4473 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/365aaba7-1b9d-463f-9231-b0edad7e1aff-istiod-ca-cert\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:09:51.021524 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:09:51.021374    4473 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3552416b-6da1-4c28-be4b-5450abc40f79-istio-data\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:09:51.021524 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:09:51.021392    4473 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/365aaba7-1b9d-463f-9231-b0edad7e1aff-credential-socket\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:09:51.021524 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:09:51.021408    4473 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/365aaba7-1b9d-463f-9231-b0edad7e1aff-workload-socket\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:09:51.021524 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:09:51.021422    4473 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3552416b-6da1-4c28-be4b-5450abc40f79-istiod-ca-cert\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:09:51.022104 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:09:51.022072    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3552416b-6da1-4c28-be4b-5450abc40f79-credential-socket" pod "3552416b-6da1-4c28-be4b-5450abc40f79" (UID: "3552416b-6da1-4c28-be4b-5450abc40f79"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:09:51.022191 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:09:51.022127    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3552416b-6da1-4c28-be4b-5450abc40f79-workload-socket" pod "3552416b-6da1-4c28-be4b-5450abc40f79" (UID: "3552416b-6da1-4c28-be4b-5450abc40f79"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:09:51.022191 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:09:51.022104    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/365aaba7-1b9d-463f-9231-b0edad7e1aff-workload-certs" pod "365aaba7-1b9d-463f-9231-b0edad7e1aff" (UID: "365aaba7-1b9d-463f-9231-b0edad7e1aff"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:09:51.022342 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:09:51.022314    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3552416b-6da1-4c28-be4b-5450abc40f79-workload-certs" pod "3552416b-6da1-4c28-be4b-5450abc40f79" (UID: "3552416b-6da1-4c28-be4b-5450abc40f79"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:09:51.022406 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:09:51.022387    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/365aaba7-1b9d-463f-9231-b0edad7e1aff-istio-data" pod "365aaba7-1b9d-463f-9231-b0edad7e1aff" (UID: "365aaba7-1b9d-463f-9231-b0edad7e1aff"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:09:51.023277 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:09:51.023241    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3552416b-6da1-4c28-be4b-5450abc40f79-istio-token" pod "3552416b-6da1-4c28-be4b-5450abc40f79" (UID: "3552416b-6da1-4c28-be4b-5450abc40f79"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:09:51.023373 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:09:51.023337    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/365aaba7-1b9d-463f-9231-b0edad7e1aff-istio-envoy" pod "365aaba7-1b9d-463f-9231-b0edad7e1aff" (UID: "365aaba7-1b9d-463f-9231-b0edad7e1aff"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:09:51.023590 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:09:51.023558    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/365aaba7-1b9d-463f-9231-b0edad7e1aff-istio-token" pod "365aaba7-1b9d-463f-9231-b0edad7e1aff" (UID: "365aaba7-1b9d-463f-9231-b0edad7e1aff"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:09:51.023698 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:09:51.023565    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3552416b-6da1-4c28-be4b-5450abc40f79-istio-podinfo" pod "3552416b-6da1-4c28-be4b-5450abc40f79" (UID: "3552416b-6da1-4c28-be4b-5450abc40f79"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:09:51.023698 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:09:51.023601    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/365aaba7-1b9d-463f-9231-b0edad7e1aff-istio-podinfo" pod "365aaba7-1b9d-463f-9231-b0edad7e1aff" (UID: "365aaba7-1b9d-463f-9231-b0edad7e1aff"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:09:51.023698 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:09:51.023655    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3552416b-6da1-4c28-be4b-5450abc40f79-istio-envoy" pod "3552416b-6da1-4c28-be4b-5450abc40f79" (UID: "3552416b-6da1-4c28-be4b-5450abc40f79"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:09:51.024622 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:09:51.024599    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3552416b-6da1-4c28-be4b-5450abc40f79-kube-api-access-n825w" pod "3552416b-6da1-4c28-be4b-5450abc40f79" (UID: "3552416b-6da1-4c28-be4b-5450abc40f79"). InnerVolumeSpecName "kube-api-access-n825w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:09:51.024713 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:09:51.024698    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/365aaba7-1b9d-463f-9231-b0edad7e1aff-kube-api-access-qg69x" pod "365aaba7-1b9d-463f-9231-b0edad7e1aff" (UID: "365aaba7-1b9d-463f-9231-b0edad7e1aff"). InnerVolumeSpecName "kube-api-access-qg69x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:09:51.123397 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:09:51.123351    4473 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3552416b-6da1-4c28-be4b-5450abc40f79-istio-token\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:09:51.123397 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:09:51.123398    4473 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/365aaba7-1b9d-463f-9231-b0edad7e1aff-istio-token\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:09:51.123662 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:09:51.123418    4473 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3552416b-6da1-4c28-be4b-5450abc40f79-istio-podinfo\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:09:51.123662 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:09:51.123433    4473 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n825w\" (UniqueName: \"kubernetes.io/projected/3552416b-6da1-4c28-be4b-5450abc40f79-kube-api-access-n825w\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:09:51.123662 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:09:51.123449    4473 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/365aaba7-1b9d-463f-9231-b0edad7e1aff-istio-envoy\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:09:51.123662 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:09:51.123462    4473 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3552416b-6da1-4c28-be4b-5450abc40f79-workload-socket\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:09:51.123662 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:09:51.123475    4473 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3552416b-6da1-4c28-be4b-5450abc40f79-istio-envoy\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:09:51.123662 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:09:51.123489    4473 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qg69x\" (UniqueName: \"kubernetes.io/projected/365aaba7-1b9d-463f-9231-b0edad7e1aff-kube-api-access-qg69x\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:09:51.123662 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:09:51.123503    4473 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3552416b-6da1-4c28-be4b-5450abc40f79-workload-certs\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:09:51.123662 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:09:51.123517    4473 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/365aaba7-1b9d-463f-9231-b0edad7e1aff-istio-data\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:09:51.123662 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:09:51.123530    4473 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3552416b-6da1-4c28-be4b-5450abc40f79-credential-socket\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:09:51.123662 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:09:51.123543    4473 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/365aaba7-1b9d-463f-9231-b0edad7e1aff-workload-certs\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:09:51.123662 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:09:51.123555    4473 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/365aaba7-1b9d-463f-9231-b0edad7e1aff-istio-podinfo\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:09:51.148067 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:09:51.148024    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-pxzmr"
Oct 30 22:09:51.148643 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:09:51.148613    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-pxzmr" event={"ID":"3552416b-6da1-4c28-be4b-5450abc40f79","Type":"ContainerDied","Data":"af7bf2b74bc83da653d00496cb706f93ddec839079216739dcb57aafa2b1ff6a"}
Oct 30 22:09:51.148723 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:09:51.148687    4473 scope.go:122] "RemoveContainer" containerID="42de881e49dd00d4bb57e44377c0ceda3c35c6e6b3cf143c396a68a2d0b4b5ec"
Oct 30 22:09:51.150001 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:09:51.149890    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-bc2x8" event={"ID":"365aaba7-1b9d-463f-9231-b0edad7e1aff","Type":"ContainerDied","Data":"410004cff69536905e4a6d5824ddaf6b8ea30988f319544f3c1f45c25e36479e"}
Oct 30 22:09:51.150115 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:09:51.150038    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-bc2x8"
Oct 30 22:09:51.163319 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:09:51.163269    4473 scope.go:122] "RemoveContainer" containerID="c1fa63726fee81fdaba4280e965b3a92b716cdccf4e7edacc7bdf749e71e2546"
Oct 30 22:09:51.344412 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:09:51.344315    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-pxzmr"]
Oct 30 22:09:51.348706 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:09:51.348678    4473 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-pxzmr"]
Oct 30 22:09:51.451875 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:09:51.451842    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-bc2x8"]
Oct 30 22:09:51.456210 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:09:51.456180    4473 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-bc2x8"]
Oct 30 22:09:52.361130 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:09:52.361095    4473 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3552416b-6da1-4c28-be4b-5450abc40f79" path="/var/lib/kubelet/pods/3552416b-6da1-4c28-be4b-5450abc40f79/volumes"
Oct 30 22:09:52.362147 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:09:52.362124    4473 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="365aaba7-1b9d-463f-9231-b0edad7e1aff" path="/var/lib/kubelet/pods/365aaba7-1b9d-463f-9231-b0edad7e1aff/volumes"
Oct 30 22:09:57.103813 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:09:57.103778    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-242ddaf0-7b7b974bc-7q9bh"
Oct 30 22:09:58.103065 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:09:58.103030    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-242ddaf0-576ccc4d89-m89lr"
Oct 30 22:10:04.237155 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:04.237118    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-90f9e9d3-6c5b7d7795-bfzhs"]
Oct 30 22:10:04.339408 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:04.339373    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-90f9e9d3-b455b57f9-rtwpf"]
Oct 30 22:10:05.277481 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:05.277441    4473 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w6-5d36f6-openshift-default-555f9495c9-9lsxw"]
Oct 30 22:10:05.278525 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:05.278493    4473 cpu_manager.go:414] "containerMap: removing container" podUID="3552416b-6da1-4c28-be4b-5450abc40f79" containerName="istio-proxy"
Oct 30 22:10:05.278525 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:05.278521    4473 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3552416b-6da1-4c28-be4b-5450abc40f79" containerName="istio-proxy"
Oct 30 22:10:05.278737 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:05.278565    4473 cpu_manager.go:414] "containerMap: removing container" podUID="365aaba7-1b9d-463f-9231-b0edad7e1aff" containerName="istio-proxy"
Oct 30 22:10:05.278737 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:05.278572    4473 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="365aaba7-1b9d-463f-9231-b0edad7e1aff" containerName="istio-proxy"
Oct 30 22:10:05.278805 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:05.278764    4473 memory_manager.go:365] "RemoveStaleState removing state" podUID="3552416b-6da1-4c28-be4b-5450abc40f79" containerName="istio-proxy"
Oct 30 22:10:05.278805 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:05.278779    4473 memory_manager.go:365] "RemoveStaleState removing state" podUID="365aaba7-1b9d-463f-9231-b0edad7e1aff" containerName="istio-proxy"
Oct 30 22:10:05.282716 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:05.282699    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w6-5d36f6-openshift-default-555f9495c9-9lsxw"
Oct 30 22:10:05.341277 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:05.341236    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w6-5d36f6-openshift-default-dockercfg-lbzzj\""
Oct 30 22:10:05.471028 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:05.470961    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/feeeb03e-f851-4d1e-bcff-0a934226a2d1-istio-envoy\") pod \"e2e-shared-a-w6-5d36f6-openshift-default-555f9495c9-9lsxw\" (UID: \"feeeb03e-f851-4d1e-bcff-0a934226a2d1\") " pod="openshift-ingress/e2e-shared-a-w6-5d36f6-openshift-default-555f9495c9-9lsxw"
Oct 30 22:10:05.481187 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:05.481142    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/feeeb03e-f851-4d1e-bcff-0a934226a2d1-istiod-ca-cert\") pod \"e2e-shared-a-w6-5d36f6-openshift-default-555f9495c9-9lsxw\" (UID: \"feeeb03e-f851-4d1e-bcff-0a934226a2d1\") " pod="openshift-ingress/e2e-shared-a-w6-5d36f6-openshift-default-555f9495c9-9lsxw"
Oct 30 22:10:05.490141 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:05.490078    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/feeeb03e-f851-4d1e-bcff-0a934226a2d1-istio-token\") pod \"e2e-shared-a-w6-5d36f6-openshift-default-555f9495c9-9lsxw\" (UID: \"feeeb03e-f851-4d1e-bcff-0a934226a2d1\") " pod="openshift-ingress/e2e-shared-a-w6-5d36f6-openshift-default-555f9495c9-9lsxw"
Oct 30 22:10:05.506163 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:05.506125    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/feeeb03e-f851-4d1e-bcff-0a934226a2d1-istio-podinfo\") pod \"e2e-shared-a-w6-5d36f6-openshift-default-555f9495c9-9lsxw\" (UID: \"feeeb03e-f851-4d1e-bcff-0a934226a2d1\") " pod="openshift-ingress/e2e-shared-a-w6-5d36f6-openshift-default-555f9495c9-9lsxw"
Oct 30 22:10:05.558075 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:05.506453    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/feeeb03e-f851-4d1e-bcff-0a934226a2d1-workload-certs\") pod \"e2e-shared-a-w6-5d36f6-openshift-default-555f9495c9-9lsxw\" (UID: \"feeeb03e-f851-4d1e-bcff-0a934226a2d1\") " pod="openshift-ingress/e2e-shared-a-w6-5d36f6-openshift-default-555f9495c9-9lsxw"
Oct 30 22:10:05.558426 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:05.558395    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wfgb4\" (UniqueName: \"kubernetes.io/projected/feeeb03e-f851-4d1e-bcff-0a934226a2d1-kube-api-access-wfgb4\") pod \"e2e-shared-a-w6-5d36f6-openshift-default-555f9495c9-9lsxw\" (UID: \"feeeb03e-f851-4d1e-bcff-0a934226a2d1\") " pod="openshift-ingress/e2e-shared-a-w6-5d36f6-openshift-default-555f9495c9-9lsxw"
Oct 30 22:10:05.558729 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:05.558709    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/feeeb03e-f851-4d1e-bcff-0a934226a2d1-credential-socket\") pod \"e2e-shared-a-w6-5d36f6-openshift-default-555f9495c9-9lsxw\" (UID: \"feeeb03e-f851-4d1e-bcff-0a934226a2d1\") " pod="openshift-ingress/e2e-shared-a-w6-5d36f6-openshift-default-555f9495c9-9lsxw"
Oct 30 22:10:05.558895 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:05.558882    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/feeeb03e-f851-4d1e-bcff-0a934226a2d1-workload-socket\") pod \"e2e-shared-a-w6-5d36f6-openshift-default-555f9495c9-9lsxw\" (UID: \"feeeb03e-f851-4d1e-bcff-0a934226a2d1\") " pod="openshift-ingress/e2e-shared-a-w6-5d36f6-openshift-default-555f9495c9-9lsxw"
Oct 30 22:10:05.559071 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:05.559056    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/feeeb03e-f851-4d1e-bcff-0a934226a2d1-istio-data\") pod \"e2e-shared-a-w6-5d36f6-openshift-default-555f9495c9-9lsxw\" (UID: \"feeeb03e-f851-4d1e-bcff-0a934226a2d1\") " pod="openshift-ingress/e2e-shared-a-w6-5d36f6-openshift-default-555f9495c9-9lsxw"
Oct 30 22:10:05.559441 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:05.559423    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w6-5d36f6-openshift-default-555f9495c9-9lsxw"]
Oct 30 22:10:05.731363 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:05.669131    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/feeeb03e-f851-4d1e-bcff-0a934226a2d1-istio-envoy\") pod \"e2e-shared-a-w6-5d36f6-openshift-default-555f9495c9-9lsxw\" (UID: \"feeeb03e-f851-4d1e-bcff-0a934226a2d1\") " pod="openshift-ingress/e2e-shared-a-w6-5d36f6-openshift-default-555f9495c9-9lsxw"
Oct 30 22:10:05.731363 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:05.669233    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/feeeb03e-f851-4d1e-bcff-0a934226a2d1-istiod-ca-cert\") pod \"e2e-shared-a-w6-5d36f6-openshift-default-555f9495c9-9lsxw\" (UID: \"feeeb03e-f851-4d1e-bcff-0a934226a2d1\") " pod="openshift-ingress/e2e-shared-a-w6-5d36f6-openshift-default-555f9495c9-9lsxw"
Oct 30 22:10:05.731363 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:05.669276    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/feeeb03e-f851-4d1e-bcff-0a934226a2d1-istio-token\") pod \"e2e-shared-a-w6-5d36f6-openshift-default-555f9495c9-9lsxw\" (UID: \"feeeb03e-f851-4d1e-bcff-0a934226a2d1\") " pod="openshift-ingress/e2e-shared-a-w6-5d36f6-openshift-default-555f9495c9-9lsxw"
Oct 30 22:10:05.731363 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:05.669378    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/feeeb03e-f851-4d1e-bcff-0a934226a2d1-istio-podinfo\") pod \"e2e-shared-a-w6-5d36f6-openshift-default-555f9495c9-9lsxw\" (UID: \"feeeb03e-f851-4d1e-bcff-0a934226a2d1\") " pod="openshift-ingress/e2e-shared-a-w6-5d36f6-openshift-default-555f9495c9-9lsxw"
Oct 30 22:10:05.731363 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:05.669426    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/feeeb03e-f851-4d1e-bcff-0a934226a2d1-workload-certs\") pod \"e2e-shared-a-w6-5d36f6-openshift-default-555f9495c9-9lsxw\" (UID: \"feeeb03e-f851-4d1e-bcff-0a934226a2d1\") " pod="openshift-ingress/e2e-shared-a-w6-5d36f6-openshift-default-555f9495c9-9lsxw"
Oct 30 22:10:05.731363 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:05.669483    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wfgb4\" (UniqueName: \"kubernetes.io/projected/feeeb03e-f851-4d1e-bcff-0a934226a2d1-kube-api-access-wfgb4\") pod \"e2e-shared-a-w6-5d36f6-openshift-default-555f9495c9-9lsxw\" (UID: \"feeeb03e-f851-4d1e-bcff-0a934226a2d1\") " pod="openshift-ingress/e2e-shared-a-w6-5d36f6-openshift-default-555f9495c9-9lsxw"
Oct 30 22:10:05.731363 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:05.669624    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/feeeb03e-f851-4d1e-bcff-0a934226a2d1-credential-socket\") pod \"e2e-shared-a-w6-5d36f6-openshift-default-555f9495c9-9lsxw\" (UID: \"feeeb03e-f851-4d1e-bcff-0a934226a2d1\") " pod="openshift-ingress/e2e-shared-a-w6-5d36f6-openshift-default-555f9495c9-9lsxw"
Oct 30 22:10:05.731363 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:05.669703    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/feeeb03e-f851-4d1e-bcff-0a934226a2d1-workload-socket\") pod \"e2e-shared-a-w6-5d36f6-openshift-default-555f9495c9-9lsxw\" (UID: \"feeeb03e-f851-4d1e-bcff-0a934226a2d1\") " pod="openshift-ingress/e2e-shared-a-w6-5d36f6-openshift-default-555f9495c9-9lsxw"
Oct 30 22:10:05.731363 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:05.669760    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/feeeb03e-f851-4d1e-bcff-0a934226a2d1-istio-data\") pod \"e2e-shared-a-w6-5d36f6-openshift-default-555f9495c9-9lsxw\" (UID: \"feeeb03e-f851-4d1e-bcff-0a934226a2d1\") " pod="openshift-ingress/e2e-shared-a-w6-5d36f6-openshift-default-555f9495c9-9lsxw"
Oct 30 22:10:05.731363 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:05.670326    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/feeeb03e-f851-4d1e-bcff-0a934226a2d1-istio-data\") pod \"e2e-shared-a-w6-5d36f6-openshift-default-555f9495c9-9lsxw\" (UID: \"feeeb03e-f851-4d1e-bcff-0a934226a2d1\") " pod="openshift-ingress/e2e-shared-a-w6-5d36f6-openshift-default-555f9495c9-9lsxw"
Oct 30 22:10:05.731363 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:05.687848    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/feeeb03e-f851-4d1e-bcff-0a934226a2d1-workload-certs\") pod \"e2e-shared-a-w6-5d36f6-openshift-default-555f9495c9-9lsxw\" (UID: \"feeeb03e-f851-4d1e-bcff-0a934226a2d1\") " pod="openshift-ingress/e2e-shared-a-w6-5d36f6-openshift-default-555f9495c9-9lsxw"
Oct 30 22:10:05.731363 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:05.688499    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/feeeb03e-f851-4d1e-bcff-0a934226a2d1-istiod-ca-cert\") pod \"e2e-shared-a-w6-5d36f6-openshift-default-555f9495c9-9lsxw\" (UID: \"feeeb03e-f851-4d1e-bcff-0a934226a2d1\") " pod="openshift-ingress/e2e-shared-a-w6-5d36f6-openshift-default-555f9495c9-9lsxw"
Oct 30 22:10:05.737878 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:05.735317    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/feeeb03e-f851-4d1e-bcff-0a934226a2d1-istio-podinfo\") pod \"e2e-shared-a-w6-5d36f6-openshift-default-555f9495c9-9lsxw\" (UID: \"feeeb03e-f851-4d1e-bcff-0a934226a2d1\") " pod="openshift-ingress/e2e-shared-a-w6-5d36f6-openshift-default-555f9495c9-9lsxw"
Oct 30 22:10:05.737878 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:05.736369    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/feeeb03e-f851-4d1e-bcff-0a934226a2d1-credential-socket\") pod \"e2e-shared-a-w6-5d36f6-openshift-default-555f9495c9-9lsxw\" (UID: \"feeeb03e-f851-4d1e-bcff-0a934226a2d1\") " pod="openshift-ingress/e2e-shared-a-w6-5d36f6-openshift-default-555f9495c9-9lsxw"
Oct 30 22:10:05.737878 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:05.737250    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/feeeb03e-f851-4d1e-bcff-0a934226a2d1-workload-socket\") pod \"e2e-shared-a-w6-5d36f6-openshift-default-555f9495c9-9lsxw\" (UID: \"feeeb03e-f851-4d1e-bcff-0a934226a2d1\") " pod="openshift-ingress/e2e-shared-a-w6-5d36f6-openshift-default-555f9495c9-9lsxw"
Oct 30 22:10:05.744753 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:05.743553    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/feeeb03e-f851-4d1e-bcff-0a934226a2d1-istio-envoy\") pod \"e2e-shared-a-w6-5d36f6-openshift-default-555f9495c9-9lsxw\" (UID: \"feeeb03e-f851-4d1e-bcff-0a934226a2d1\") " pod="openshift-ingress/e2e-shared-a-w6-5d36f6-openshift-default-555f9495c9-9lsxw"
Oct 30 22:10:05.748149 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:05.747449    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/feeeb03e-f851-4d1e-bcff-0a934226a2d1-istio-token\") pod \"e2e-shared-a-w6-5d36f6-openshift-default-555f9495c9-9lsxw\" (UID: \"feeeb03e-f851-4d1e-bcff-0a934226a2d1\") " pod="openshift-ingress/e2e-shared-a-w6-5d36f6-openshift-default-555f9495c9-9lsxw"
Oct 30 22:10:06.009951 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:06.007100    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wfgb4\" (UniqueName: \"kubernetes.io/projected/feeeb03e-f851-4d1e-bcff-0a934226a2d1-kube-api-access-wfgb4\") pod \"e2e-shared-a-w6-5d36f6-openshift-default-555f9495c9-9lsxw\" (UID: \"feeeb03e-f851-4d1e-bcff-0a934226a2d1\") " pod="openshift-ingress/e2e-shared-a-w6-5d36f6-openshift-default-555f9495c9-9lsxw"
Oct 30 22:10:06.213202 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:06.213164    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w6-5d36f6-openshift-default-555f9495c9-9lsxw"
Oct 30 22:10:07.305236 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:07.305179    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w6-5d36f6-openshift-default-555f9495c9-9lsxw"]
Oct 30 22:10:07.331608 ip-10-0-140-18 kubenswrapper[4473]: W1030 22:10:07.331570    4473 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfeeeb03e_f851_4d1e_bcff_0a934226a2d1.slice/crio-9314b21c4f9914d3d127c4226768cc5c56c9751221acff604745551e228235a4 WatchSource:0}: Error finding container 9314b21c4f9914d3d127c4226768cc5c56c9751221acff604745551e228235a4: Status 404 returned error can't find the container with id 9314b21c4f9914d3d127c4226768cc5c56c9751221acff604745551e228235a4
Oct 30 22:10:07.334626 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:07.334587    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:10:07.334750 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:07.334679    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:10:07.334750 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:07.334725    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:10:08.347975 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:08.347930    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w6-5d36f6-openshift-default-555f9495c9-9lsxw" event={"ID":"feeeb03e-f851-4d1e-bcff-0a934226a2d1","Type":"ContainerStarted","Data":"271a8613be6a58fa1d39fb6218ab6cb2dd30ddfb473d04c6bc8618966a2f20bd"}
Oct 30 22:10:08.348353 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:08.347987    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w6-5d36f6-openshift-default-555f9495c9-9lsxw" event={"ID":"feeeb03e-f851-4d1e-bcff-0a934226a2d1","Type":"ContainerStarted","Data":"9314b21c4f9914d3d127c4226768cc5c56c9751221acff604745551e228235a4"}
Oct 30 22:10:08.575726 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:08.575673    4473 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w6-5d36f6-openshift-default-555f9495c9-9lsxw" podStartSLOduration=3.575653139 podStartE2EDuration="3.575653139s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:10:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:10:08.523562188 +0000 UTC m=+1654.296930679" watchObservedRunningTime="2026-10-30 22:10:08.575653139 +0000 UTC m=+1654.349021623"
Oct 30 22:10:09.214534 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:09.214494    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w6-5d36f6-openshift-default-555f9495c9-9lsxw"
Oct 30 22:10:09.220676 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:09.220650    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w6-5d36f6-openshift-default-555f9495c9-9lsxw"
Oct 30 22:10:09.237981 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:09.237914    4473 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-migrate-90f9e9d3-6c5b7d7795-bfzhs" podUID="a81a41ac-1fa3-4416-b4ac-fb9ee9bee450" containerName="payload-processing" containerID="cri-o://8582794f38ff6184fe669d2af76eae39771f3aba98a4c7b4303a91340c4461b0" gracePeriod=25
Oct 30 22:10:09.340685 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:09.340638    4473 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-migrate-90f9e9d3-b455b57f9-rtwpf" podUID="cfa1c7a1-06ad-4ea2-b6b7-fa0a1d04cdc7" containerName="payload-pre-processing" containerID="cri-o://0bd0b505fe21a806ea08b4c592cfe25d6e62748930c32b96a827968977708810" gracePeriod=25
Oct 30 22:10:09.358199 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:09.358161    4473 generic.go:356] "Generic (PLEG): container finished" podID="a81a41ac-1fa3-4416-b4ac-fb9ee9bee450" containerID="8582794f38ff6184fe669d2af76eae39771f3aba98a4c7b4303a91340c4461b0" exitCode=0
Oct 30 22:10:09.358681 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:09.358229    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-90f9e9d3-6c5b7d7795-bfzhs" event={"ID":"a81a41ac-1fa3-4416-b4ac-fb9ee9bee450","Type":"ContainerDied","Data":"8582794f38ff6184fe669d2af76eae39771f3aba98a4c7b4303a91340c4461b0"}
Oct 30 22:10:09.358753 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:09.358694    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w6-5d36f6-openshift-default-555f9495c9-9lsxw"
Oct 30 22:10:09.359895 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:09.359874    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w6-5d36f6-openshift-default-555f9495c9-9lsxw"
Oct 30 22:10:09.561068 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:09.561039    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-90f9e9d3-6c5b7d7795-bfzhs"
Oct 30 22:10:09.614069 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:09.611275    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a81a41ac-1fa3-4416-b4ac-fb9ee9bee450-kube-api-access-78trv\" (UniqueName: \"kubernetes.io/projected/a81a41ac-1fa3-4416-b4ac-fb9ee9bee450-kube-api-access-78trv\") pod \"a81a41ac-1fa3-4416-b4ac-fb9ee9bee450\" (UID: \"a81a41ac-1fa3-4416-b4ac-fb9ee9bee450\") "
Oct 30 22:10:09.614069 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:09.611338    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a81a41ac-1fa3-4416-b4ac-fb9ee9bee450-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a81a41ac-1fa3-4416-b4ac-fb9ee9bee450-plugins-config-volume\") pod \"a81a41ac-1fa3-4416-b4ac-fb9ee9bee450\" (UID: \"a81a41ac-1fa3-4416-b4ac-fb9ee9bee450\") "
Oct 30 22:10:09.614069 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:09.612133    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a81a41ac-1fa3-4416-b4ac-fb9ee9bee450-plugins-config-volume" pod "a81a41ac-1fa3-4416-b4ac-fb9ee9bee450" (UID: "a81a41ac-1fa3-4416-b4ac-fb9ee9bee450"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:10:09.614069 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:09.612414    4473 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a81a41ac-1fa3-4416-b4ac-fb9ee9bee450-plugins-config-volume\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:10:09.615846 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:09.615801    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a81a41ac-1fa3-4416-b4ac-fb9ee9bee450-kube-api-access-78trv" pod "a81a41ac-1fa3-4416-b4ac-fb9ee9bee450" (UID: "a81a41ac-1fa3-4416-b4ac-fb9ee9bee450"). InnerVolumeSpecName "kube-api-access-78trv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:10:09.687308 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:09.687278    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-90f9e9d3-b455b57f9-rtwpf"
Oct 30 22:10:09.714204 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:09.714170    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cfa1c7a1-06ad-4ea2-b6b7-fa0a1d04cdc7-kube-api-access-ql2c8\" (UniqueName: \"kubernetes.io/projected/cfa1c7a1-06ad-4ea2-b6b7-fa0a1d04cdc7-kube-api-access-ql2c8\") pod \"cfa1c7a1-06ad-4ea2-b6b7-fa0a1d04cdc7\" (UID: \"cfa1c7a1-06ad-4ea2-b6b7-fa0a1d04cdc7\") "
Oct 30 22:10:09.714425 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:09.714243    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/cfa1c7a1-06ad-4ea2-b6b7-fa0a1d04cdc7-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cfa1c7a1-06ad-4ea2-b6b7-fa0a1d04cdc7-plugins-config-volume\") pod \"cfa1c7a1-06ad-4ea2-b6b7-fa0a1d04cdc7\" (UID: \"cfa1c7a1-06ad-4ea2-b6b7-fa0a1d04cdc7\") "
Oct 30 22:10:09.714712 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:09.714686    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cfa1c7a1-06ad-4ea2-b6b7-fa0a1d04cdc7-plugins-config-volume" pod "cfa1c7a1-06ad-4ea2-b6b7-fa0a1d04cdc7" (UID: "cfa1c7a1-06ad-4ea2-b6b7-fa0a1d04cdc7"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:10:09.714967 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:09.714938    4473 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-78trv\" (UniqueName: \"kubernetes.io/projected/a81a41ac-1fa3-4416-b4ac-fb9ee9bee450-kube-api-access-78trv\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:10:09.714967 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:09.714963    4473 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cfa1c7a1-06ad-4ea2-b6b7-fa0a1d04cdc7-plugins-config-volume\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:10:09.716724 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:09.716693    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cfa1c7a1-06ad-4ea2-b6b7-fa0a1d04cdc7-kube-api-access-ql2c8" pod "cfa1c7a1-06ad-4ea2-b6b7-fa0a1d04cdc7" (UID: "cfa1c7a1-06ad-4ea2-b6b7-fa0a1d04cdc7"). InnerVolumeSpecName "kube-api-access-ql2c8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:10:09.816732 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:09.816645    4473 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ql2c8\" (UniqueName: \"kubernetes.io/projected/cfa1c7a1-06ad-4ea2-b6b7-fa0a1d04cdc7-kube-api-access-ql2c8\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:10:10.368762 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:10.368730    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-90f9e9d3-6c5b7d7795-bfzhs"
Oct 30 22:10:10.368762 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:10.368745    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-90f9e9d3-6c5b7d7795-bfzhs" event={"ID":"a81a41ac-1fa3-4416-b4ac-fb9ee9bee450","Type":"ContainerDied","Data":"97f01cf98eaec24800f0955b94cc0b65d64dca1393e3a9dd35519743848301a9"}
Oct 30 22:10:10.369303 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:10.368813    4473 scope.go:122] "RemoveContainer" containerID="8582794f38ff6184fe669d2af76eae39771f3aba98a4c7b4303a91340c4461b0"
Oct 30 22:10:10.370513 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:10.370450    4473 generic.go:356] "Generic (PLEG): container finished" podID="cfa1c7a1-06ad-4ea2-b6b7-fa0a1d04cdc7" containerID="0bd0b505fe21a806ea08b4c592cfe25d6e62748930c32b96a827968977708810" exitCode=0
Oct 30 22:10:10.370513 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:10.370503    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-90f9e9d3-b455b57f9-rtwpf"
Oct 30 22:10:10.370778 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:10.370535    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-90f9e9d3-b455b57f9-rtwpf" event={"ID":"cfa1c7a1-06ad-4ea2-b6b7-fa0a1d04cdc7","Type":"ContainerDied","Data":"0bd0b505fe21a806ea08b4c592cfe25d6e62748930c32b96a827968977708810"}
Oct 30 22:10:10.370778 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:10.370589    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-90f9e9d3-b455b57f9-rtwpf" event={"ID":"cfa1c7a1-06ad-4ea2-b6b7-fa0a1d04cdc7","Type":"ContainerDied","Data":"2288d07bbdc9c17aa914fc008566d2240cdddd3b714562d68ebd1a9f2aefded4"}
Oct 30 22:10:10.384535 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:10.384511    4473 scope.go:122] "RemoveContainer" containerID="0bd0b505fe21a806ea08b4c592cfe25d6e62748930c32b96a827968977708810"
Oct 30 22:10:10.395352 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:10.395327    4473 scope.go:122] "RemoveContainer" containerID="0bd0b505fe21a806ea08b4c592cfe25d6e62748930c32b96a827968977708810"
Oct 30 22:10:10.395710 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:10:10.395690    4473 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0bd0b505fe21a806ea08b4c592cfe25d6e62748930c32b96a827968977708810\": container with ID starting with 0bd0b505fe21a806ea08b4c592cfe25d6e62748930c32b96a827968977708810 not found: ID does not exist" containerID="0bd0b505fe21a806ea08b4c592cfe25d6e62748930c32b96a827968977708810"
Oct 30 22:10:10.395767 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:10.395724    4473 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0bd0b505fe21a806ea08b4c592cfe25d6e62748930c32b96a827968977708810"} err="failed to get container status \"0bd0b505fe21a806ea08b4c592cfe25d6e62748930c32b96a827968977708810\": rpc error: code = NotFound desc = could not find container \"0bd0b505fe21a806ea08b4c592cfe25d6e62748930c32b96a827968977708810\": container with ID starting with 0bd0b505fe21a806ea08b4c592cfe25d6e62748930c32b96a827968977708810 not found: ID does not exist"
Oct 30 22:10:10.538194 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:10.538104    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-90f9e9d3-6c5b7d7795-bfzhs"]
Oct 30 22:10:10.542279 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:10.542252    4473 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-90f9e9d3-6c5b7d7795-bfzhs"]
Oct 30 22:10:10.653322 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:10.653274    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-90f9e9d3-b455b57f9-rtwpf"]
Oct 30 22:10:10.658552 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:10.658518    4473 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-90f9e9d3-b455b57f9-rtwpf"]
Oct 30 22:10:12.362187 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:12.362155    4473 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a81a41ac-1fa3-4416-b4ac-fb9ee9bee450" path="/var/lib/kubelet/pods/a81a41ac-1fa3-4416-b4ac-fb9ee9bee450/volumes"
Oct 30 22:10:12.362652 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:12.362635    4473 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="cfa1c7a1-06ad-4ea2-b6b7-fa0a1d04cdc7" path="/var/lib/kubelet/pods/cfa1c7a1-06ad-4ea2-b6b7-fa0a1d04cdc7/volumes"
Oct 30 22:10:13.511767 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:13.511731    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-6r982"]
Oct 30 22:10:13.512235 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:13.512084    4473 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-6r982" podUID="48aa5363-fb6c-4cfe-a76d-e8aec041e46f" containerName="istio-proxy" containerID="cri-o://e96649e74ccb53a7dd66222b8dc1373b948644c41d56f4fae45a6d98e6a082d3" gracePeriod=30
Oct 30 22:10:13.561888 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:13.561839    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-bs4lr"]
Oct 30 22:10:13.562211 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:13.562182    4473 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-bs4lr" podUID="335708bc-083d-4ce1-9158-cc61dcac93da" containerName="istio-proxy" containerID="cri-o://3fe63c4f9af714c3502e0939e441ec7caaa4ce6fec3f1467b36f57facc47ddf8" gracePeriod=30
Oct 30 22:10:13.566216 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:13.566186    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-xrf95"]
Oct 30 22:10:13.566409 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:13.566388    4473 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-xrf95" podUID="ef9eca59-6c6f-4523-9046-4fd0611dacef" containerName="istio-proxy" containerID="cri-o://16a4e0937aac7ea2443ad5fdf740a27f8021688faa118d0eaf746bd60f4229ca" gracePeriod=30
Oct 30 22:10:15.409057 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:15.408170    4473 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-8f1d78db-openshift-default-6bcc897b9-r5blj"]
Oct 30 22:10:15.462943 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:15.437351    4473 cpu_manager.go:414] "containerMap: removing container" podUID="a81a41ac-1fa3-4416-b4ac-fb9ee9bee450" containerName="payload-processing"
Oct 30 22:10:15.463196 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:15.463174    4473 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a81a41ac-1fa3-4416-b4ac-fb9ee9bee450" containerName="payload-processing"
Oct 30 22:10:15.580152 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:15.580125    4473 cpu_manager.go:414] "containerMap: removing container" podUID="cfa1c7a1-06ad-4ea2-b6b7-fa0a1d04cdc7" containerName="payload-pre-processing"
Oct 30 22:10:15.590006 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:15.589970    4473 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cfa1c7a1-06ad-4ea2-b6b7-fa0a1d04cdc7" containerName="payload-pre-processing"
Oct 30 22:10:15.609196 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:15.609143    4473 memory_manager.go:365] "RemoveStaleState removing state" podUID="cfa1c7a1-06ad-4ea2-b6b7-fa0a1d04cdc7" containerName="payload-pre-processing"
Oct 30 22:10:15.609413 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:15.609206    4473 memory_manager.go:365] "RemoveStaleState removing state" podUID="a81a41ac-1fa3-4416-b4ac-fb9ee9bee450" containerName="payload-processing"
Oct 30 22:10:15.732258 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:15.732179    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-8f1d78db-openshift-default-6bcc897b9-r5blj"]
Oct 30 22:10:15.732619 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:15.732542    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-8f1d78db-openshift-default-6bcc897b9-r5blj"
Oct 30 22:10:15.821124 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:15.821064    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3e4b6a7f-8bfd-49d8-be83-674d091c99d4-istio-data\") pod \"e2e-mt-8f1d78db-openshift-default-6bcc897b9-r5blj\" (UID: \"3e4b6a7f-8bfd-49d8-be83-674d091c99d4\") " pod="openshift-ingress/e2e-mt-8f1d78db-openshift-default-6bcc897b9-r5blj"
Oct 30 22:10:15.821429 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:15.821406    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3e4b6a7f-8bfd-49d8-be83-674d091c99d4-workload-certs\") pod \"e2e-mt-8f1d78db-openshift-default-6bcc897b9-r5blj\" (UID: \"3e4b6a7f-8bfd-49d8-be83-674d091c99d4\") " pod="openshift-ingress/e2e-mt-8f1d78db-openshift-default-6bcc897b9-r5blj"
Oct 30 22:10:15.821620 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:15.821606    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e4b6a7f-8bfd-49d8-be83-674d091c99d4-workload-socket\") pod \"e2e-mt-8f1d78db-openshift-default-6bcc897b9-r5blj\" (UID: \"3e4b6a7f-8bfd-49d8-be83-674d091c99d4\") " pod="openshift-ingress/e2e-mt-8f1d78db-openshift-default-6bcc897b9-r5blj"
Oct 30 22:10:15.821728 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:15.821715    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3e4b6a7f-8bfd-49d8-be83-674d091c99d4-istio-token\") pod \"e2e-mt-8f1d78db-openshift-default-6bcc897b9-r5blj\" (UID: \"3e4b6a7f-8bfd-49d8-be83-674d091c99d4\") " pod="openshift-ingress/e2e-mt-8f1d78db-openshift-default-6bcc897b9-r5blj"
Oct 30 22:10:15.822142 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:15.822117    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3e4b6a7f-8bfd-49d8-be83-674d091c99d4-istio-envoy\") pod \"e2e-mt-8f1d78db-openshift-default-6bcc897b9-r5blj\" (UID: \"3e4b6a7f-8bfd-49d8-be83-674d091c99d4\") " pod="openshift-ingress/e2e-mt-8f1d78db-openshift-default-6bcc897b9-r5blj"
Oct 30 22:10:15.822307 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:15.822288    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3e4b6a7f-8bfd-49d8-be83-674d091c99d4-istiod-ca-cert\") pod \"e2e-mt-8f1d78db-openshift-default-6bcc897b9-r5blj\" (UID: \"3e4b6a7f-8bfd-49d8-be83-674d091c99d4\") " pod="openshift-ingress/e2e-mt-8f1d78db-openshift-default-6bcc897b9-r5blj"
Oct 30 22:10:15.822474 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:15.822450    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j4247\" (UniqueName: \"kubernetes.io/projected/3e4b6a7f-8bfd-49d8-be83-674d091c99d4-kube-api-access-j4247\") pod \"e2e-mt-8f1d78db-openshift-default-6bcc897b9-r5blj\" (UID: \"3e4b6a7f-8bfd-49d8-be83-674d091c99d4\") " pod="openshift-ingress/e2e-mt-8f1d78db-openshift-default-6bcc897b9-r5blj"
Oct 30 22:10:15.822640 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:15.822603    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e4b6a7f-8bfd-49d8-be83-674d091c99d4-credential-socket\") pod \"e2e-mt-8f1d78db-openshift-default-6bcc897b9-r5blj\" (UID: \"3e4b6a7f-8bfd-49d8-be83-674d091c99d4\") " pod="openshift-ingress/e2e-mt-8f1d78db-openshift-default-6bcc897b9-r5blj"
Oct 30 22:10:15.822792 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:15.822777    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3e4b6a7f-8bfd-49d8-be83-674d091c99d4-istio-podinfo\") pod \"e2e-mt-8f1d78db-openshift-default-6bcc897b9-r5blj\" (UID: \"3e4b6a7f-8bfd-49d8-be83-674d091c99d4\") " pod="openshift-ingress/e2e-mt-8f1d78db-openshift-default-6bcc897b9-r5blj"
Oct 30 22:10:15.836964 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:15.836840    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-8f1d78db-openshift-default-dockercfg-splff\""
Oct 30 22:10:15.924736 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:15.924687    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3e4b6a7f-8bfd-49d8-be83-674d091c99d4-istio-envoy\") pod \"e2e-mt-8f1d78db-openshift-default-6bcc897b9-r5blj\" (UID: \"3e4b6a7f-8bfd-49d8-be83-674d091c99d4\") " pod="openshift-ingress/e2e-mt-8f1d78db-openshift-default-6bcc897b9-r5blj"
Oct 30 22:10:15.924975 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:15.924745    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3e4b6a7f-8bfd-49d8-be83-674d091c99d4-istiod-ca-cert\") pod \"e2e-mt-8f1d78db-openshift-default-6bcc897b9-r5blj\" (UID: \"3e4b6a7f-8bfd-49d8-be83-674d091c99d4\") " pod="openshift-ingress/e2e-mt-8f1d78db-openshift-default-6bcc897b9-r5blj"
Oct 30 22:10:15.924975 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:15.924794    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j4247\" (UniqueName: \"kubernetes.io/projected/3e4b6a7f-8bfd-49d8-be83-674d091c99d4-kube-api-access-j4247\") pod \"e2e-mt-8f1d78db-openshift-default-6bcc897b9-r5blj\" (UID: \"3e4b6a7f-8bfd-49d8-be83-674d091c99d4\") " pod="openshift-ingress/e2e-mt-8f1d78db-openshift-default-6bcc897b9-r5blj"
Oct 30 22:10:15.924975 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:15.924825    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e4b6a7f-8bfd-49d8-be83-674d091c99d4-credential-socket\") pod \"e2e-mt-8f1d78db-openshift-default-6bcc897b9-r5blj\" (UID: \"3e4b6a7f-8bfd-49d8-be83-674d091c99d4\") " pod="openshift-ingress/e2e-mt-8f1d78db-openshift-default-6bcc897b9-r5blj"
Oct 30 22:10:15.924975 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:15.924894    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3e4b6a7f-8bfd-49d8-be83-674d091c99d4-istio-podinfo\") pod \"e2e-mt-8f1d78db-openshift-default-6bcc897b9-r5blj\" (UID: \"3e4b6a7f-8bfd-49d8-be83-674d091c99d4\") " pod="openshift-ingress/e2e-mt-8f1d78db-openshift-default-6bcc897b9-r5blj"
Oct 30 22:10:15.925233 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:15.925123    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3e4b6a7f-8bfd-49d8-be83-674d091c99d4-istio-data\") pod \"e2e-mt-8f1d78db-openshift-default-6bcc897b9-r5blj\" (UID: \"3e4b6a7f-8bfd-49d8-be83-674d091c99d4\") " pod="openshift-ingress/e2e-mt-8f1d78db-openshift-default-6bcc897b9-r5blj"
Oct 30 22:10:15.925233 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:15.925225    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3e4b6a7f-8bfd-49d8-be83-674d091c99d4-workload-certs\") pod \"e2e-mt-8f1d78db-openshift-default-6bcc897b9-r5blj\" (UID: \"3e4b6a7f-8bfd-49d8-be83-674d091c99d4\") " pod="openshift-ingress/e2e-mt-8f1d78db-openshift-default-6bcc897b9-r5blj"
Oct 30 22:10:15.925369 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:15.925332    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e4b6a7f-8bfd-49d8-be83-674d091c99d4-workload-socket\") pod \"e2e-mt-8f1d78db-openshift-default-6bcc897b9-r5blj\" (UID: \"3e4b6a7f-8bfd-49d8-be83-674d091c99d4\") " pod="openshift-ingress/e2e-mt-8f1d78db-openshift-default-6bcc897b9-r5blj"
Oct 30 22:10:15.925369 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:15.925223    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e4b6a7f-8bfd-49d8-be83-674d091c99d4-credential-socket\") pod \"e2e-mt-8f1d78db-openshift-default-6bcc897b9-r5blj\" (UID: \"3e4b6a7f-8bfd-49d8-be83-674d091c99d4\") " pod="openshift-ingress/e2e-mt-8f1d78db-openshift-default-6bcc897b9-r5blj"
Oct 30 22:10:15.925369 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:15.925364    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3e4b6a7f-8bfd-49d8-be83-674d091c99d4-istio-token\") pod \"e2e-mt-8f1d78db-openshift-default-6bcc897b9-r5blj\" (UID: \"3e4b6a7f-8bfd-49d8-be83-674d091c99d4\") " pod="openshift-ingress/e2e-mt-8f1d78db-openshift-default-6bcc897b9-r5blj"
Oct 30 22:10:15.925529 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:15.925433    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3e4b6a7f-8bfd-49d8-be83-674d091c99d4-istio-data\") pod \"e2e-mt-8f1d78db-openshift-default-6bcc897b9-r5blj\" (UID: \"3e4b6a7f-8bfd-49d8-be83-674d091c99d4\") " pod="openshift-ingress/e2e-mt-8f1d78db-openshift-default-6bcc897b9-r5blj"
Oct 30 22:10:15.925529 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:15.925443    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3e4b6a7f-8bfd-49d8-be83-674d091c99d4-workload-certs\") pod \"e2e-mt-8f1d78db-openshift-default-6bcc897b9-r5blj\" (UID: \"3e4b6a7f-8bfd-49d8-be83-674d091c99d4\") " pod="openshift-ingress/e2e-mt-8f1d78db-openshift-default-6bcc897b9-r5blj"
Oct 30 22:10:15.925840 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:15.925822    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3e4b6a7f-8bfd-49d8-be83-674d091c99d4-istiod-ca-cert\") pod \"e2e-mt-8f1d78db-openshift-default-6bcc897b9-r5blj\" (UID: \"3e4b6a7f-8bfd-49d8-be83-674d091c99d4\") " pod="openshift-ingress/e2e-mt-8f1d78db-openshift-default-6bcc897b9-r5blj"
Oct 30 22:10:15.925840 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:15.925826    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e4b6a7f-8bfd-49d8-be83-674d091c99d4-workload-socket\") pod \"e2e-mt-8f1d78db-openshift-default-6bcc897b9-r5blj\" (UID: \"3e4b6a7f-8bfd-49d8-be83-674d091c99d4\") " pod="openshift-ingress/e2e-mt-8f1d78db-openshift-default-6bcc897b9-r5blj"
Oct 30 22:10:15.927471 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:15.927449    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3e4b6a7f-8bfd-49d8-be83-674d091c99d4-istio-envoy\") pod \"e2e-mt-8f1d78db-openshift-default-6bcc897b9-r5blj\" (UID: \"3e4b6a7f-8bfd-49d8-be83-674d091c99d4\") " pod="openshift-ingress/e2e-mt-8f1d78db-openshift-default-6bcc897b9-r5blj"
Oct 30 22:10:15.927889 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:15.927868    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3e4b6a7f-8bfd-49d8-be83-674d091c99d4-istio-podinfo\") pod \"e2e-mt-8f1d78db-openshift-default-6bcc897b9-r5blj\" (UID: \"3e4b6a7f-8bfd-49d8-be83-674d091c99d4\") " pod="openshift-ingress/e2e-mt-8f1d78db-openshift-default-6bcc897b9-r5blj"
Oct 30 22:10:15.983785 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:15.983707    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3e4b6a7f-8bfd-49d8-be83-674d091c99d4-istio-token\") pod \"e2e-mt-8f1d78db-openshift-default-6bcc897b9-r5blj\" (UID: \"3e4b6a7f-8bfd-49d8-be83-674d091c99d4\") " pod="openshift-ingress/e2e-mt-8f1d78db-openshift-default-6bcc897b9-r5blj"
Oct 30 22:10:15.983942 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:15.983790    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j4247\" (UniqueName: \"kubernetes.io/projected/3e4b6a7f-8bfd-49d8-be83-674d091c99d4-kube-api-access-j4247\") pod \"e2e-mt-8f1d78db-openshift-default-6bcc897b9-r5blj\" (UID: \"3e4b6a7f-8bfd-49d8-be83-674d091c99d4\") " pod="openshift-ingress/e2e-mt-8f1d78db-openshift-default-6bcc897b9-r5blj"
Oct 30 22:10:16.115665 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:16.115621    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-8f1d78db-openshift-default-6bcc897b9-r5blj"
Oct 30 22:10:16.552980 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:16.552937    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-8f1d78db-openshift-default-6bcc897b9-r5blj"]
Oct 30 22:10:16.588987 ip-10-0-140-18 kubenswrapper[4473]: W1030 22:10:16.588957    4473 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e4b6a7f_8bfd_49d8_be83_674d091c99d4.slice/crio-a09ace75c64f3898543a2476d7fb7d0b946519493530ba3ab304a772d22ae09a WatchSource:0}: Error finding container a09ace75c64f3898543a2476d7fb7d0b946519493530ba3ab304a772d22ae09a: Status 404 returned error can't find the container with id a09ace75c64f3898543a2476d7fb7d0b946519493530ba3ab304a772d22ae09a
Oct 30 22:10:16.591780 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:16.591744    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:10:16.591857 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:16.591816    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:10:16.591894 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:16.591866    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:10:17.553580 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:17.553507    4473 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w6-5d36f6-757f4b4d99-mrs2n"]
Oct 30 22:10:17.585679 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:17.569170    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w6-5d36f6-757f4b4d99-mrs2n"
Oct 30 22:10:17.586097 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:17.586063    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-8f1d78db-openshift-default-6bcc897b9-r5blj" event={"ID":"3e4b6a7f-8bfd-49d8-be83-674d091c99d4","Type":"ContainerStarted","Data":"99754613fc190c15476ba88cb6724abf21129e7aa43928ceaee60e7bc615d280"}
Oct 30 22:10:17.586235 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:17.586129    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-8f1d78db-openshift-default-6bcc897b9-r5blj" event={"ID":"3e4b6a7f-8bfd-49d8-be83-674d091c99d4","Type":"ContainerStarted","Data":"a09ace75c64f3898543a2476d7fb7d0b946519493530ba3ab304a772d22ae09a"}
Oct 30 22:10:17.600741 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:17.600698    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w6-5d36f6-757f4b4d99-mrs2n"]
Oct 30 22:10:17.681952 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:17.680612    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w6-5d36f6-dockercfg-lwhpk\""
Oct 30 22:10:17.691084 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:17.683064    4473 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w6-5d36f6\""
Oct 30 22:10:17.738117 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:17.734985    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2810c233-3078-4bc0-bb3e-bc863e7786d2-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w6-5d36f6-757f4b4d99-mrs2n\" (UID: \"2810c233-3078-4bc0-bb3e-bc863e7786d2\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w6-5d36f6-757f4b4d99-mrs2n"
Oct 30 22:10:17.738117 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:17.735128    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f852v\" (UniqueName: \"kubernetes.io/projected/2810c233-3078-4bc0-bb3e-bc863e7786d2-kube-api-access-f852v\") pod \"payload-pre-processing-e2e-shared-a-w6-5d36f6-757f4b4d99-mrs2n\" (UID: \"2810c233-3078-4bc0-bb3e-bc863e7786d2\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w6-5d36f6-757f4b4d99-mrs2n"
Oct 30 22:10:17.944073 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:17.843306    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2810c233-3078-4bc0-bb3e-bc863e7786d2-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w6-5d36f6-757f4b4d99-mrs2n\" (UID: \"2810c233-3078-4bc0-bb3e-bc863e7786d2\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w6-5d36f6-757f4b4d99-mrs2n"
Oct 30 22:10:18.012942 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:17.853994    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2810c233-3078-4bc0-bb3e-bc863e7786d2-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w6-5d36f6-757f4b4d99-mrs2n\" (UID: \"2810c233-3078-4bc0-bb3e-bc863e7786d2\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w6-5d36f6-757f4b4d99-mrs2n"
Oct 30 22:10:18.097150 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:17.894533    4473 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w6-5d36f6-84c6b5685c-zjsct"]
Oct 30 22:10:18.153989 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:18.046266    4473 patch_prober.go:29] interesting pod/e2e-worker-w1-d516c4-openshift-default-6bd584745-xrf95 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 22:10:18.153989 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:18.152621    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-xrf95" podUID="ef9eca59-6c6f-4523-9046-4fd0611dacef" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 22:10:18.153989 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:18.107091    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f852v\" (UniqueName: \"kubernetes.io/projected/2810c233-3078-4bc0-bb3e-bc863e7786d2-kube-api-access-f852v\") pod \"payload-pre-processing-e2e-shared-a-w6-5d36f6-757f4b4d99-mrs2n\" (UID: \"2810c233-3078-4bc0-bb3e-bc863e7786d2\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w6-5d36f6-757f4b4d99-mrs2n"
Oct 30 22:10:18.182079 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:18.167156    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w6-5d36f6-84c6b5685c-zjsct"
Oct 30 22:10:18.183097 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:18.183072    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-8f1d78db-openshift-default-6bcc897b9-r5blj"
Oct 30 22:10:18.246138 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:18.246056    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-8f1d78db-openshift-default-6bcc897b9-r5blj"
Oct 30 22:10:18.301958 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:18.301893    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w6-5d36f6-84c6b5685c-zjsct"]
Oct 30 22:10:18.323556 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:18.310679    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f852v\" (UniqueName: \"kubernetes.io/projected/2810c233-3078-4bc0-bb3e-bc863e7786d2-kube-api-access-f852v\") pod \"payload-pre-processing-e2e-shared-a-w6-5d36f6-757f4b4d99-mrs2n\" (UID: \"2810c233-3078-4bc0-bb3e-bc863e7786d2\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w6-5d36f6-757f4b4d99-mrs2n"
Oct 30 22:10:18.358792 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:18.358719    4473 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-8f1d78db-openshift-default-6bcc897b9-r5blj" podStartSLOduration=3.358696505 podStartE2EDuration="3.358696505s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:10:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:10:17.894460758 +0000 UTC m=+1663.667829251" watchObservedRunningTime="2026-10-30 22:10:18.358696505 +0000 UTC m=+1664.132064992"
Oct 30 22:10:18.383155 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:18.383095    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b69j9\" (UniqueName: \"kubernetes.io/projected/b72da9e3-1c30-4fb9-a879-1279c67a7453-kube-api-access-b69j9\") pod \"payload-processing-e2e-shared-a-w6-5d36f6-84c6b5685c-zjsct\" (UID: \"b72da9e3-1c30-4fb9-a879-1279c67a7453\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w6-5d36f6-84c6b5685c-zjsct"
Oct 30 22:10:18.383504 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:18.383481    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b72da9e3-1c30-4fb9-a879-1279c67a7453-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w6-5d36f6-84c6b5685c-zjsct\" (UID: \"b72da9e3-1c30-4fb9-a879-1279c67a7453\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w6-5d36f6-84c6b5685c-zjsct"
Oct 30 22:10:18.488049 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:18.486881    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b69j9\" (UniqueName: \"kubernetes.io/projected/b72da9e3-1c30-4fb9-a879-1279c67a7453-kube-api-access-b69j9\") pod \"payload-processing-e2e-shared-a-w6-5d36f6-84c6b5685c-zjsct\" (UID: \"b72da9e3-1c30-4fb9-a879-1279c67a7453\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w6-5d36f6-84c6b5685c-zjsct"
Oct 30 22:10:18.488354 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:18.488316    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b72da9e3-1c30-4fb9-a879-1279c67a7453-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w6-5d36f6-84c6b5685c-zjsct\" (UID: \"b72da9e3-1c30-4fb9-a879-1279c67a7453\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w6-5d36f6-84c6b5685c-zjsct"
Oct 30 22:10:18.490237 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:18.489996    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b72da9e3-1c30-4fb9-a879-1279c67a7453-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w6-5d36f6-84c6b5685c-zjsct\" (UID: \"b72da9e3-1c30-4fb9-a879-1279c67a7453\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w6-5d36f6-84c6b5685c-zjsct"
Oct 30 22:10:18.547005 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:18.536622    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w6-5d36f6-757f4b4d99-mrs2n"
Oct 30 22:10:18.550948 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:18.550052    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b69j9\" (UniqueName: \"kubernetes.io/projected/b72da9e3-1c30-4fb9-a879-1279c67a7453-kube-api-access-b69j9\") pod \"payload-processing-e2e-shared-a-w6-5d36f6-84c6b5685c-zjsct\" (UID: \"b72da9e3-1c30-4fb9-a879-1279c67a7453\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w6-5d36f6-84c6b5685c-zjsct"
Oct 30 22:10:18.595619 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:18.595589    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-8f1d78db-openshift-default-6bcc897b9-r5blj"
Oct 30 22:10:18.596687 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:18.596668    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-8f1d78db-openshift-default-6bcc897b9-r5blj"
Oct 30 22:10:18.768241 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:18.767973    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w6-5d36f6-84c6b5685c-zjsct"
Oct 30 22:10:18.937029 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:18.937000    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-6r982"
Oct 30 22:10:18.977988 ip-10-0-140-18 kubenswrapper[4473]: W1030 22:10:18.977326    4473 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2810c233_3078_4bc0_bb3e_bc863e7786d2.slice/crio-f102c7eac367dc1592c6ac95c61716ea1de275dc29a7975057474ea86a92e553 WatchSource:0}: Error finding container f102c7eac367dc1592c6ac95c61716ea1de275dc29a7975057474ea86a92e553: Status 404 returned error can't find the container with id f102c7eac367dc1592c6ac95c61716ea1de275dc29a7975057474ea86a92e553
Oct 30 22:10:18.980741 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:18.980717    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-xrf95"
Oct 30 22:10:18.982086 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:18.981994    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w6-5d36f6-757f4b4d99-mrs2n"]
Oct 30 22:10:18.985124 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:18.985105    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-bs4lr"
Oct 30 22:10:19.007024 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:19.006876    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/48aa5363-fb6c-4cfe-a76d-e8aec041e46f-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/48aa5363-fb6c-4cfe-a76d-e8aec041e46f-istio-podinfo\") pod \"48aa5363-fb6c-4cfe-a76d-e8aec041e46f\" (UID: \"48aa5363-fb6c-4cfe-a76d-e8aec041e46f\") "
Oct 30 22:10:19.007024 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:19.006956    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/48aa5363-fb6c-4cfe-a76d-e8aec041e46f-kube-api-access-kh2bx\" (UniqueName: \"kubernetes.io/projected/48aa5363-fb6c-4cfe-a76d-e8aec041e46f-kube-api-access-kh2bx\") pod \"48aa5363-fb6c-4cfe-a76d-e8aec041e46f\" (UID: \"48aa5363-fb6c-4cfe-a76d-e8aec041e46f\") "
Oct 30 22:10:19.007024 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:19.006996    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ef9eca59-6c6f-4523-9046-4fd0611dacef-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ef9eca59-6c6f-4523-9046-4fd0611dacef-credential-socket\") pod \"ef9eca59-6c6f-4523-9046-4fd0611dacef\" (UID: \"ef9eca59-6c6f-4523-9046-4fd0611dacef\") "
Oct 30 22:10:19.007608 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:19.007039    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/335708bc-083d-4ce1-9158-cc61dcac93da-istio-token\" (UniqueName: \"kubernetes.io/projected/335708bc-083d-4ce1-9158-cc61dcac93da-istio-token\") pod \"335708bc-083d-4ce1-9158-cc61dcac93da\" (UID: \"335708bc-083d-4ce1-9158-cc61dcac93da\") "
Oct 30 22:10:19.007608 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:19.007116    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/48aa5363-fb6c-4cfe-a76d-e8aec041e46f-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/48aa5363-fb6c-4cfe-a76d-e8aec041e46f-credential-socket\") pod \"48aa5363-fb6c-4cfe-a76d-e8aec041e46f\" (UID: \"48aa5363-fb6c-4cfe-a76d-e8aec041e46f\") "
Oct 30 22:10:19.007608 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:19.007204    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/48aa5363-fb6c-4cfe-a76d-e8aec041e46f-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/48aa5363-fb6c-4cfe-a76d-e8aec041e46f-istio-envoy\") pod \"48aa5363-fb6c-4cfe-a76d-e8aec041e46f\" (UID: \"48aa5363-fb6c-4cfe-a76d-e8aec041e46f\") "
Oct 30 22:10:19.007608 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:19.007230    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/ef9eca59-6c6f-4523-9046-4fd0611dacef-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ef9eca59-6c6f-4523-9046-4fd0611dacef-istio-podinfo\") pod \"ef9eca59-6c6f-4523-9046-4fd0611dacef\" (UID: \"ef9eca59-6c6f-4523-9046-4fd0611dacef\") "
Oct 30 22:10:19.007608 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:19.007279    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ef9eca59-6c6f-4523-9046-4fd0611dacef-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ef9eca59-6c6f-4523-9046-4fd0611dacef-workload-socket\") pod \"ef9eca59-6c6f-4523-9046-4fd0611dacef\" (UID: \"ef9eca59-6c6f-4523-9046-4fd0611dacef\") "
Oct 30 22:10:19.007608 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:19.007312    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/335708bc-083d-4ce1-9158-cc61dcac93da-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/335708bc-083d-4ce1-9158-cc61dcac93da-istio-podinfo\") pod \"335708bc-083d-4ce1-9158-cc61dcac93da\" (UID: \"335708bc-083d-4ce1-9158-cc61dcac93da\") "
Oct 30 22:10:19.007608 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:19.007429    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ef9eca59-6c6f-4523-9046-4fd0611dacef-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ef9eca59-6c6f-4523-9046-4fd0611dacef-istio-envoy\") pod \"ef9eca59-6c6f-4523-9046-4fd0611dacef\" (UID: \"ef9eca59-6c6f-4523-9046-4fd0611dacef\") "
Oct 30 22:10:19.007608 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:19.007481    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/48aa5363-fb6c-4cfe-a76d-e8aec041e46f-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/48aa5363-fb6c-4cfe-a76d-e8aec041e46f-workload-certs\") pod \"48aa5363-fb6c-4cfe-a76d-e8aec041e46f\" (UID: \"48aa5363-fb6c-4cfe-a76d-e8aec041e46f\") "
Oct 30 22:10:19.007608 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:19.007561    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ef9eca59-6c6f-4523-9046-4fd0611dacef-istio-token\" (UniqueName: \"kubernetes.io/projected/ef9eca59-6c6f-4523-9046-4fd0611dacef-istio-token\") pod \"ef9eca59-6c6f-4523-9046-4fd0611dacef\" (UID: \"ef9eca59-6c6f-4523-9046-4fd0611dacef\") "
Oct 30 22:10:19.008090 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:19.007614    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/48aa5363-fb6c-4cfe-a76d-e8aec041e46f-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/48aa5363-fb6c-4cfe-a76d-e8aec041e46f-workload-socket\") pod \"48aa5363-fb6c-4cfe-a76d-e8aec041e46f\" (UID: \"48aa5363-fb6c-4cfe-a76d-e8aec041e46f\") "
Oct 30 22:10:19.008090 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:19.007680    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/48aa5363-fb6c-4cfe-a76d-e8aec041e46f-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/48aa5363-fb6c-4cfe-a76d-e8aec041e46f-istio-data\") pod \"48aa5363-fb6c-4cfe-a76d-e8aec041e46f\" (UID: \"48aa5363-fb6c-4cfe-a76d-e8aec041e46f\") "
Oct 30 22:10:19.008090 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:19.007704    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/335708bc-083d-4ce1-9158-cc61dcac93da-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/335708bc-083d-4ce1-9158-cc61dcac93da-workload-certs\") pod \"335708bc-083d-4ce1-9158-cc61dcac93da\" (UID: \"335708bc-083d-4ce1-9158-cc61dcac93da\") "
Oct 30 22:10:19.008090 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:19.007771    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ef9eca59-6c6f-4523-9046-4fd0611dacef-kube-api-access-mm9mm\" (UniqueName: \"kubernetes.io/projected/ef9eca59-6c6f-4523-9046-4fd0611dacef-kube-api-access-mm9mm\") pod \"ef9eca59-6c6f-4523-9046-4fd0611dacef\" (UID: \"ef9eca59-6c6f-4523-9046-4fd0611dacef\") "
Oct 30 22:10:19.008090 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:19.007814    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/335708bc-083d-4ce1-9158-cc61dcac93da-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/335708bc-083d-4ce1-9158-cc61dcac93da-istio-data\") pod \"335708bc-083d-4ce1-9158-cc61dcac93da\" (UID: \"335708bc-083d-4ce1-9158-cc61dcac93da\") "
Oct 30 22:10:19.008090 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:19.007853    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/48aa5363-fb6c-4cfe-a76d-e8aec041e46f-istio-token\" (UniqueName: \"kubernetes.io/projected/48aa5363-fb6c-4cfe-a76d-e8aec041e46f-istio-token\") pod \"48aa5363-fb6c-4cfe-a76d-e8aec041e46f\" (UID: \"48aa5363-fb6c-4cfe-a76d-e8aec041e46f\") "
Oct 30 22:10:19.008090 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:19.007882    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ef9eca59-6c6f-4523-9046-4fd0611dacef-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ef9eca59-6c6f-4523-9046-4fd0611dacef-istio-data\") pod \"ef9eca59-6c6f-4523-9046-4fd0611dacef\" (UID: \"ef9eca59-6c6f-4523-9046-4fd0611dacef\") "
Oct 30 22:10:19.008090 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:19.007992    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/48aa5363-fb6c-4cfe-a76d-e8aec041e46f-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/48aa5363-fb6c-4cfe-a76d-e8aec041e46f-istiod-ca-cert\") pod \"48aa5363-fb6c-4cfe-a76d-e8aec041e46f\" (UID: \"48aa5363-fb6c-4cfe-a76d-e8aec041e46f\") "
Oct 30 22:10:19.008090 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:19.008089    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/335708bc-083d-4ce1-9158-cc61dcac93da-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/335708bc-083d-4ce1-9158-cc61dcac93da-credential-socket\") pod \"335708bc-083d-4ce1-9158-cc61dcac93da\" (UID: \"335708bc-083d-4ce1-9158-cc61dcac93da\") "
Oct 30 22:10:19.008607 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:19.008153    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/335708bc-083d-4ce1-9158-cc61dcac93da-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/335708bc-083d-4ce1-9158-cc61dcac93da-istio-envoy\") pod \"335708bc-083d-4ce1-9158-cc61dcac93da\" (UID: \"335708bc-083d-4ce1-9158-cc61dcac93da\") "
Oct 30 22:10:19.008607 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:19.008250    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ef9eca59-6c6f-4523-9046-4fd0611dacef-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ef9eca59-6c6f-4523-9046-4fd0611dacef-workload-certs\") pod \"ef9eca59-6c6f-4523-9046-4fd0611dacef\" (UID: \"ef9eca59-6c6f-4523-9046-4fd0611dacef\") "
Oct 30 22:10:19.008607 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:19.008281    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/335708bc-083d-4ce1-9158-cc61dcac93da-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/335708bc-083d-4ce1-9158-cc61dcac93da-istiod-ca-cert\") pod \"335708bc-083d-4ce1-9158-cc61dcac93da\" (UID: \"335708bc-083d-4ce1-9158-cc61dcac93da\") "
Oct 30 22:10:19.008607 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:19.008349    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/335708bc-083d-4ce1-9158-cc61dcac93da-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/335708bc-083d-4ce1-9158-cc61dcac93da-workload-socket\") pod \"335708bc-083d-4ce1-9158-cc61dcac93da\" (UID: \"335708bc-083d-4ce1-9158-cc61dcac93da\") "
Oct 30 22:10:19.008607 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:19.008397    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/335708bc-083d-4ce1-9158-cc61dcac93da-kube-api-access-n865n\" (UniqueName: \"kubernetes.io/projected/335708bc-083d-4ce1-9158-cc61dcac93da-kube-api-access-n865n\") pod \"335708bc-083d-4ce1-9158-cc61dcac93da\" (UID: \"335708bc-083d-4ce1-9158-cc61dcac93da\") "
Oct 30 22:10:19.008607 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:19.008415    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ef9eca59-6c6f-4523-9046-4fd0611dacef-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ef9eca59-6c6f-4523-9046-4fd0611dacef-istiod-ca-cert\") pod \"ef9eca59-6c6f-4523-9046-4fd0611dacef\" (UID: \"ef9eca59-6c6f-4523-9046-4fd0611dacef\") "
Oct 30 22:10:19.009483 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:19.009447    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ef9eca59-6c6f-4523-9046-4fd0611dacef-istiod-ca-cert" pod "ef9eca59-6c6f-4523-9046-4fd0611dacef" (UID: "ef9eca59-6c6f-4523-9046-4fd0611dacef"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:10:19.009771 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:19.009747    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/335708bc-083d-4ce1-9158-cc61dcac93da-workload-certs" pod "335708bc-083d-4ce1-9158-cc61dcac93da" (UID: "335708bc-083d-4ce1-9158-cc61dcac93da"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:10:19.009882 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:19.009810    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ef9eca59-6c6f-4523-9046-4fd0611dacef-credential-socket" pod "ef9eca59-6c6f-4523-9046-4fd0611dacef" (UID: "ef9eca59-6c6f-4523-9046-4fd0611dacef"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:10:19.010548 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:19.010214    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/48aa5363-fb6c-4cfe-a76d-e8aec041e46f-kube-api-access-kh2bx" pod "48aa5363-fb6c-4cfe-a76d-e8aec041e46f" (UID: "48aa5363-fb6c-4cfe-a76d-e8aec041e46f"). InnerVolumeSpecName "kube-api-access-kh2bx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:10:19.011223 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:19.010953    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/335708bc-083d-4ce1-9158-cc61dcac93da-istio-data" pod "335708bc-083d-4ce1-9158-cc61dcac93da" (UID: "335708bc-083d-4ce1-9158-cc61dcac93da"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:10:19.011728 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:19.011455    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ef9eca59-6c6f-4523-9046-4fd0611dacef-istio-data" pod "ef9eca59-6c6f-4523-9046-4fd0611dacef" (UID: "ef9eca59-6c6f-4523-9046-4fd0611dacef"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:10:19.012757 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:19.012479    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/335708bc-083d-4ce1-9158-cc61dcac93da-istiod-ca-cert" pod "335708bc-083d-4ce1-9158-cc61dcac93da" (UID: "335708bc-083d-4ce1-9158-cc61dcac93da"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:10:19.012757 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:19.012515    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/48aa5363-fb6c-4cfe-a76d-e8aec041e46f-istiod-ca-cert" pod "48aa5363-fb6c-4cfe-a76d-e8aec041e46f" (UID: "48aa5363-fb6c-4cfe-a76d-e8aec041e46f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:10:19.013198 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:19.013001    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ef9eca59-6c6f-4523-9046-4fd0611dacef-workload-certs" pod "ef9eca59-6c6f-4523-9046-4fd0611dacef" (UID: "ef9eca59-6c6f-4523-9046-4fd0611dacef"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:10:19.013537 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:19.013338    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/48aa5363-fb6c-4cfe-a76d-e8aec041e46f-credential-socket" pod "48aa5363-fb6c-4cfe-a76d-e8aec041e46f" (UID: "48aa5363-fb6c-4cfe-a76d-e8aec041e46f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:10:19.013537 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:19.013398    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/335708bc-083d-4ce1-9158-cc61dcac93da-workload-socket" pod "335708bc-083d-4ce1-9158-cc61dcac93da" (UID: "335708bc-083d-4ce1-9158-cc61dcac93da"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:10:19.014612 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:19.013989    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/48aa5363-fb6c-4cfe-a76d-e8aec041e46f-workload-socket" pod "48aa5363-fb6c-4cfe-a76d-e8aec041e46f" (UID: "48aa5363-fb6c-4cfe-a76d-e8aec041e46f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:10:19.014612 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:19.014177    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/48aa5363-fb6c-4cfe-a76d-e8aec041e46f-istio-data" pod "48aa5363-fb6c-4cfe-a76d-e8aec041e46f" (UID: "48aa5363-fb6c-4cfe-a76d-e8aec041e46f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:10:19.014612 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:19.014240    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/335708bc-083d-4ce1-9158-cc61dcac93da-istio-podinfo" pod "335708bc-083d-4ce1-9158-cc61dcac93da" (UID: "335708bc-083d-4ce1-9158-cc61dcac93da"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:10:19.014904 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:19.014871    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ef9eca59-6c6f-4523-9046-4fd0611dacef-workload-socket" pod "ef9eca59-6c6f-4523-9046-4fd0611dacef" (UID: "ef9eca59-6c6f-4523-9046-4fd0611dacef"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:10:19.015164 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:19.015142    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/335708bc-083d-4ce1-9158-cc61dcac93da-credential-socket" pod "335708bc-083d-4ce1-9158-cc61dcac93da" (UID: "335708bc-083d-4ce1-9158-cc61dcac93da"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:10:19.015360 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:19.015326    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/48aa5363-fb6c-4cfe-a76d-e8aec041e46f-workload-certs" pod "48aa5363-fb6c-4cfe-a76d-e8aec041e46f" (UID: "48aa5363-fb6c-4cfe-a76d-e8aec041e46f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:10:19.016969 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:19.016945    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ef9eca59-6c6f-4523-9046-4fd0611dacef-istio-podinfo" pod "ef9eca59-6c6f-4523-9046-4fd0611dacef" (UID: "ef9eca59-6c6f-4523-9046-4fd0611dacef"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:10:19.017181 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:19.017163    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ef9eca59-6c6f-4523-9046-4fd0611dacef-istio-envoy" pod "ef9eca59-6c6f-4523-9046-4fd0611dacef" (UID: "ef9eca59-6c6f-4523-9046-4fd0611dacef"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:10:19.017261 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:19.017227    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ef9eca59-6c6f-4523-9046-4fd0611dacef-istio-token" pod "ef9eca59-6c6f-4523-9046-4fd0611dacef" (UID: "ef9eca59-6c6f-4523-9046-4fd0611dacef"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:10:19.018076 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:19.018046    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/335708bc-083d-4ce1-9158-cc61dcac93da-kube-api-access-n865n" pod "335708bc-083d-4ce1-9158-cc61dcac93da" (UID: "335708bc-083d-4ce1-9158-cc61dcac93da"). InnerVolumeSpecName "kube-api-access-n865n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:10:19.018191 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:19.018172    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/335708bc-083d-4ce1-9158-cc61dcac93da-istio-envoy" pod "335708bc-083d-4ce1-9158-cc61dcac93da" (UID: "335708bc-083d-4ce1-9158-cc61dcac93da"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:10:19.018263 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:19.018199    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/335708bc-083d-4ce1-9158-cc61dcac93da-istio-token" pod "335708bc-083d-4ce1-9158-cc61dcac93da" (UID: "335708bc-083d-4ce1-9158-cc61dcac93da"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:10:19.018351 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:19.018330    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/48aa5363-fb6c-4cfe-a76d-e8aec041e46f-istio-podinfo" pod "48aa5363-fb6c-4cfe-a76d-e8aec041e46f" (UID: "48aa5363-fb6c-4cfe-a76d-e8aec041e46f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:10:19.018402 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:19.018364    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ef9eca59-6c6f-4523-9046-4fd0611dacef-kube-api-access-mm9mm" pod "ef9eca59-6c6f-4523-9046-4fd0611dacef" (UID: "ef9eca59-6c6f-4523-9046-4fd0611dacef"). InnerVolumeSpecName "kube-api-access-mm9mm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:10:19.019380 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:19.019352    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/48aa5363-fb6c-4cfe-a76d-e8aec041e46f-istio-envoy" pod "48aa5363-fb6c-4cfe-a76d-e8aec041e46f" (UID: "48aa5363-fb6c-4cfe-a76d-e8aec041e46f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:10:19.020212 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:19.020179    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/48aa5363-fb6c-4cfe-a76d-e8aec041e46f-istio-token" pod "48aa5363-fb6c-4cfe-a76d-e8aec041e46f" (UID: "48aa5363-fb6c-4cfe-a76d-e8aec041e46f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:10:19.110903 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:19.110809    4473 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/335708bc-083d-4ce1-9158-cc61dcac93da-istio-envoy\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:10:19.110903 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:19.110849    4473 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ef9eca59-6c6f-4523-9046-4fd0611dacef-workload-certs\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:10:19.110903 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:19.110860    4473 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/335708bc-083d-4ce1-9158-cc61dcac93da-istiod-ca-cert\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:10:19.110903 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:19.110869    4473 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/335708bc-083d-4ce1-9158-cc61dcac93da-workload-socket\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:10:19.110903 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:19.110879    4473 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n865n\" (UniqueName: \"kubernetes.io/projected/335708bc-083d-4ce1-9158-cc61dcac93da-kube-api-access-n865n\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:10:19.110903 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:19.110890    4473 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ef9eca59-6c6f-4523-9046-4fd0611dacef-istiod-ca-cert\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:10:19.110903 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:19.110899    4473 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/48aa5363-fb6c-4cfe-a76d-e8aec041e46f-istio-podinfo\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:10:19.110903 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:19.110909    4473 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kh2bx\" (UniqueName: \"kubernetes.io/projected/48aa5363-fb6c-4cfe-a76d-e8aec041e46f-kube-api-access-kh2bx\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:10:19.111301 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:19.110935    4473 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ef9eca59-6c6f-4523-9046-4fd0611dacef-credential-socket\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:10:19.111301 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:19.110948    4473 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/335708bc-083d-4ce1-9158-cc61dcac93da-istio-token\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:10:19.111301 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:19.110960    4473 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/48aa5363-fb6c-4cfe-a76d-e8aec041e46f-credential-socket\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:10:19.111301 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:19.110971    4473 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/48aa5363-fb6c-4cfe-a76d-e8aec041e46f-istio-envoy\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:10:19.111301 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:19.110980    4473 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ef9eca59-6c6f-4523-9046-4fd0611dacef-istio-podinfo\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:10:19.111301 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:19.110989    4473 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ef9eca59-6c6f-4523-9046-4fd0611dacef-workload-socket\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:10:19.111301 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:19.110997    4473 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/335708bc-083d-4ce1-9158-cc61dcac93da-istio-podinfo\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:10:19.111301 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:19.111005    4473 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ef9eca59-6c6f-4523-9046-4fd0611dacef-istio-envoy\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:10:19.111301 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:19.111013    4473 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/48aa5363-fb6c-4cfe-a76d-e8aec041e46f-workload-certs\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:10:19.111301 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:19.111021    4473 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ef9eca59-6c6f-4523-9046-4fd0611dacef-istio-token\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:10:19.111301 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:19.111030    4473 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/48aa5363-fb6c-4cfe-a76d-e8aec041e46f-workload-socket\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:10:19.111301 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:19.111039    4473 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/48aa5363-fb6c-4cfe-a76d-e8aec041e46f-istio-data\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:10:19.111301 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:19.111047    4473 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/335708bc-083d-4ce1-9158-cc61dcac93da-workload-certs\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:10:19.111301 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:19.111059    4473 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mm9mm\" (UniqueName: \"kubernetes.io/projected/ef9eca59-6c6f-4523-9046-4fd0611dacef-kube-api-access-mm9mm\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:10:19.111301 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:19.111071    4473 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/335708bc-083d-4ce1-9158-cc61dcac93da-istio-data\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:10:19.111301 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:19.111079    4473 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/48aa5363-fb6c-4cfe-a76d-e8aec041e46f-istio-token\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:10:19.111301 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:19.111088    4473 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ef9eca59-6c6f-4523-9046-4fd0611dacef-istio-data\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:10:19.111301 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:19.111097    4473 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/48aa5363-fb6c-4cfe-a76d-e8aec041e46f-istiod-ca-cert\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:10:19.111301 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:19.111106    4473 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/335708bc-083d-4ce1-9158-cc61dcac93da-credential-socket\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:10:19.223869 ip-10-0-140-18 kubenswrapper[4473]: W1030 22:10:19.223833    4473 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb72da9e3_1c30_4fb9_a879_1279c67a7453.slice/crio-86b0aa016b4a7ead5dbf393b84b69e13456f4409eadecc3e4658efa59979d88e WatchSource:0}: Error finding container 86b0aa016b4a7ead5dbf393b84b69e13456f4409eadecc3e4658efa59979d88e: Status 404 returned error can't find the container with id 86b0aa016b4a7ead5dbf393b84b69e13456f4409eadecc3e4658efa59979d88e
Oct 30 22:10:19.227255 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:19.227221    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w6-5d36f6-84c6b5685c-zjsct"]
Oct 30 22:10:19.605619 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:19.605586    4473 generic.go:356] "Generic (PLEG): container finished" podID="ef9eca59-6c6f-4523-9046-4fd0611dacef" containerID="16a4e0937aac7ea2443ad5fdf740a27f8021688faa118d0eaf746bd60f4229ca" exitCode=0
Oct 30 22:10:19.606132 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:19.605670    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-xrf95"
Oct 30 22:10:19.606132 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:19.605669    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-xrf95" event={"ID":"ef9eca59-6c6f-4523-9046-4fd0611dacef","Type":"ContainerDied","Data":"16a4e0937aac7ea2443ad5fdf740a27f8021688faa118d0eaf746bd60f4229ca"}
Oct 30 22:10:19.606132 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:19.605735    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-xrf95" event={"ID":"ef9eca59-6c6f-4523-9046-4fd0611dacef","Type":"ContainerDied","Data":"320cee48b509e6c096395caf16e940e026cadf470cca761e3aae132d084643bd"}
Oct 30 22:10:19.606132 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:19.605772    4473 scope.go:122] "RemoveContainer" containerID="16a4e0937aac7ea2443ad5fdf740a27f8021688faa118d0eaf746bd60f4229ca"
Oct 30 22:10:19.607898 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:19.607873    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w6-5d36f6-84c6b5685c-zjsct" event={"ID":"b72da9e3-1c30-4fb9-a879-1279c67a7453","Type":"ContainerStarted","Data":"2d0ef0b34a04084a97c378fc1accfc726888fa1bfe96cffeb1b0678f3425344a"}
Oct 30 22:10:19.608141 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:19.607950    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w6-5d36f6-84c6b5685c-zjsct"
Oct 30 22:10:19.608141 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:19.607976    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w6-5d36f6-84c6b5685c-zjsct" event={"ID":"b72da9e3-1c30-4fb9-a879-1279c67a7453","Type":"ContainerStarted","Data":"86b0aa016b4a7ead5dbf393b84b69e13456f4409eadecc3e4658efa59979d88e"}
Oct 30 22:10:19.609874 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:19.609854    4473 generic.go:356] "Generic (PLEG): container finished" podID="335708bc-083d-4ce1-9158-cc61dcac93da" containerID="3fe63c4f9af714c3502e0939e441ec7caaa4ce6fec3f1467b36f57facc47ddf8" exitCode=0
Oct 30 22:10:19.610009 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:19.609941    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-bs4lr"
Oct 30 22:10:19.610009 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:19.609962    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-bs4lr" event={"ID":"335708bc-083d-4ce1-9158-cc61dcac93da","Type":"ContainerDied","Data":"3fe63c4f9af714c3502e0939e441ec7caaa4ce6fec3f1467b36f57facc47ddf8"}
Oct 30 22:10:19.610124 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:19.610018    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-bs4lr" event={"ID":"335708bc-083d-4ce1-9158-cc61dcac93da","Type":"ContainerDied","Data":"b905be3188056fdd6bccfff53c5254be59a0cd70fa04a8df940ea3d5adc15e07"}
Oct 30 22:10:19.611681 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:19.611653    4473 generic.go:356] "Generic (PLEG): container finished" podID="48aa5363-fb6c-4cfe-a76d-e8aec041e46f" containerID="e96649e74ccb53a7dd66222b8dc1373b948644c41d56f4fae45a6d98e6a082d3" exitCode=0
Oct 30 22:10:19.611779 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:19.611680    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-6r982" event={"ID":"48aa5363-fb6c-4cfe-a76d-e8aec041e46f","Type":"ContainerDied","Data":"e96649e74ccb53a7dd66222b8dc1373b948644c41d56f4fae45a6d98e6a082d3"}
Oct 30 22:10:19.611779 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:19.611737    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-6r982" event={"ID":"48aa5363-fb6c-4cfe-a76d-e8aec041e46f","Type":"ContainerDied","Data":"ab7528d99c6a2572c34322a6ec4de3f95fbcec5e1e9c51250b5cd49d1792ccb5"}
Oct 30 22:10:19.611779 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:19.611697    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-6r982"
Oct 30 22:10:19.613691 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:19.613665    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w6-5d36f6-757f4b4d99-mrs2n" event={"ID":"2810c233-3078-4bc0-bb3e-bc863e7786d2","Type":"ContainerStarted","Data":"6454d5d1253d3fd12a27db76a7970982c6e6f74aae1343667872ba420b03d185"}
Oct 30 22:10:19.613802 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:19.613719    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w6-5d36f6-757f4b4d99-mrs2n" event={"ID":"2810c233-3078-4bc0-bb3e-bc863e7786d2","Type":"ContainerStarted","Data":"f102c7eac367dc1592c6ac95c61716ea1de275dc29a7975057474ea86a92e553"}
Oct 30 22:10:19.613958 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:19.613942    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w6-5d36f6-757f4b4d99-mrs2n"
Oct 30 22:10:19.620666 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:19.620646    4473 scope.go:122] "RemoveContainer" containerID="16a4e0937aac7ea2443ad5fdf740a27f8021688faa118d0eaf746bd60f4229ca"
Oct 30 22:10:19.621093 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:10:19.621069    4473 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"16a4e0937aac7ea2443ad5fdf740a27f8021688faa118d0eaf746bd60f4229ca\": container with ID starting with 16a4e0937aac7ea2443ad5fdf740a27f8021688faa118d0eaf746bd60f4229ca not found: ID does not exist" containerID="16a4e0937aac7ea2443ad5fdf740a27f8021688faa118d0eaf746bd60f4229ca"
Oct 30 22:10:19.621165 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:19.621109    4473 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"16a4e0937aac7ea2443ad5fdf740a27f8021688faa118d0eaf746bd60f4229ca"} err="failed to get container status \"16a4e0937aac7ea2443ad5fdf740a27f8021688faa118d0eaf746bd60f4229ca\": rpc error: code = NotFound desc = could not find container \"16a4e0937aac7ea2443ad5fdf740a27f8021688faa118d0eaf746bd60f4229ca\": container with ID starting with 16a4e0937aac7ea2443ad5fdf740a27f8021688faa118d0eaf746bd60f4229ca not found: ID does not exist"
Oct 30 22:10:19.621165 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:19.621135    4473 scope.go:122] "RemoveContainer" containerID="3fe63c4f9af714c3502e0939e441ec7caaa4ce6fec3f1467b36f57facc47ddf8"
Oct 30 22:10:19.635874 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:19.635851    4473 scope.go:122] "RemoveContainer" containerID="f8e22d2edf99ab821c2b98a87d18effbb6bb9b17a0e4ac6ec2693fd3826b85dd"
Oct 30 22:10:19.649707 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:19.649687    4473 scope.go:122] "RemoveContainer" containerID="3fe63c4f9af714c3502e0939e441ec7caaa4ce6fec3f1467b36f57facc47ddf8"
Oct 30 22:10:19.650043 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:10:19.650022    4473 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3fe63c4f9af714c3502e0939e441ec7caaa4ce6fec3f1467b36f57facc47ddf8\": container with ID starting with 3fe63c4f9af714c3502e0939e441ec7caaa4ce6fec3f1467b36f57facc47ddf8 not found: ID does not exist" containerID="3fe63c4f9af714c3502e0939e441ec7caaa4ce6fec3f1467b36f57facc47ddf8"
Oct 30 22:10:19.650160 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:19.650059    4473 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3fe63c4f9af714c3502e0939e441ec7caaa4ce6fec3f1467b36f57facc47ddf8"} err="failed to get container status \"3fe63c4f9af714c3502e0939e441ec7caaa4ce6fec3f1467b36f57facc47ddf8\": rpc error: code = NotFound desc = could not find container \"3fe63c4f9af714c3502e0939e441ec7caaa4ce6fec3f1467b36f57facc47ddf8\": container with ID starting with 3fe63c4f9af714c3502e0939e441ec7caaa4ce6fec3f1467b36f57facc47ddf8 not found: ID does not exist"
Oct 30 22:10:19.650160 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:19.650086    4473 scope.go:122] "RemoveContainer" containerID="f8e22d2edf99ab821c2b98a87d18effbb6bb9b17a0e4ac6ec2693fd3826b85dd"
Oct 30 22:10:19.650397 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:10:19.650374    4473 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f8e22d2edf99ab821c2b98a87d18effbb6bb9b17a0e4ac6ec2693fd3826b85dd\": container with ID starting with f8e22d2edf99ab821c2b98a87d18effbb6bb9b17a0e4ac6ec2693fd3826b85dd not found: ID does not exist" containerID="f8e22d2edf99ab821c2b98a87d18effbb6bb9b17a0e4ac6ec2693fd3826b85dd"
Oct 30 22:10:19.650438 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:19.650404    4473 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f8e22d2edf99ab821c2b98a87d18effbb6bb9b17a0e4ac6ec2693fd3826b85dd"} err="failed to get container status \"f8e22d2edf99ab821c2b98a87d18effbb6bb9b17a0e4ac6ec2693fd3826b85dd\": rpc error: code = NotFound desc = could not find container \"f8e22d2edf99ab821c2b98a87d18effbb6bb9b17a0e4ac6ec2693fd3826b85dd\": container with ID starting with f8e22d2edf99ab821c2b98a87d18effbb6bb9b17a0e4ac6ec2693fd3826b85dd not found: ID does not exist"
Oct 30 22:10:19.650438 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:19.650419    4473 scope.go:122] "RemoveContainer" containerID="e96649e74ccb53a7dd66222b8dc1373b948644c41d56f4fae45a6d98e6a082d3"
Oct 30 22:10:19.661813 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:19.661795    4473 scope.go:122] "RemoveContainer" containerID="e96649e74ccb53a7dd66222b8dc1373b948644c41d56f4fae45a6d98e6a082d3"
Oct 30 22:10:19.662194 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:10:19.662171    4473 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e96649e74ccb53a7dd66222b8dc1373b948644c41d56f4fae45a6d98e6a082d3\": container with ID starting with e96649e74ccb53a7dd66222b8dc1373b948644c41d56f4fae45a6d98e6a082d3 not found: ID does not exist" containerID="e96649e74ccb53a7dd66222b8dc1373b948644c41d56f4fae45a6d98e6a082d3"
Oct 30 22:10:19.662277 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:19.662202    4473 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e96649e74ccb53a7dd66222b8dc1373b948644c41d56f4fae45a6d98e6a082d3"} err="failed to get container status \"e96649e74ccb53a7dd66222b8dc1373b948644c41d56f4fae45a6d98e6a082d3\": rpc error: code = NotFound desc = could not find container \"e96649e74ccb53a7dd66222b8dc1373b948644c41d56f4fae45a6d98e6a082d3\": container with ID starting with e96649e74ccb53a7dd66222b8dc1373b948644c41d56f4fae45a6d98e6a082d3 not found: ID does not exist"
Oct 30 22:10:19.777114 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:19.777081    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-xrf95"]
Oct 30 22:10:19.781329 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:19.781305    4473 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-xrf95"]
Oct 30 22:10:19.897454 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:19.897391    4473 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w6-5d36f6-84c6b5685c-zjsct" podStartSLOduration=2.897375349 podStartE2EDuration="2.897375349s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:10:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:10:19.892186673 +0000 UTC m=+1665.665555160" watchObservedRunningTime="2026-10-30 22:10:19.897375349 +0000 UTC m=+1665.670743906"
Oct 30 22:10:20.013570 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:20.013517    4473 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w6-5d36f6-757f4b4d99-mrs2n" podStartSLOduration=3.013504863 podStartE2EDuration="3.013504863s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:10:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:10:20.01221974 +0000 UTC m=+1665.785588229" watchObservedRunningTime="2026-10-30 22:10:20.013504863 +0000 UTC m=+1665.786873349"
Oct 30 22:10:20.158006 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:20.131906    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-6r982"]
Oct 30 22:10:20.194404 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:20.194362    4473 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-6r982"]
Oct 30 22:10:20.267676 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:20.267329    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-bs4lr"]
Oct 30 22:10:20.274178 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:20.274145    4473 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-bs4lr"]
Oct 30 22:10:20.327411 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:10:20.327378    4473 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fncvg" containerName="extractor"
Oct 30 22:10:20.360224 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:20.360193    4473 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="335708bc-083d-4ce1-9158-cc61dcac93da" path="/var/lib/kubelet/pods/335708bc-083d-4ce1-9158-cc61dcac93da/volumes"
Oct 30 22:10:20.360883 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:20.360867    4473 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="48aa5363-fb6c-4cfe-a76d-e8aec041e46f" path="/var/lib/kubelet/pods/48aa5363-fb6c-4cfe-a76d-e8aec041e46f/volumes"
Oct 30 22:10:20.361351 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:20.361338    4473 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ef9eca59-6c6f-4523-9046-4fd0611dacef" path="/var/lib/kubelet/pods/ef9eca59-6c6f-4523-9046-4fd0611dacef/volumes"
Oct 30 22:10:24.821824 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:24.821787    4473 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-8f1d78db-6cf8f64455-rhrhn"]
Oct 30 22:10:24.825104 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:24.824419    4473 cpu_manager.go:414] "containerMap: removing container" podUID="48aa5363-fb6c-4cfe-a76d-e8aec041e46f" containerName="istio-proxy"
Oct 30 22:10:24.825104 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:24.824450    4473 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="48aa5363-fb6c-4cfe-a76d-e8aec041e46f" containerName="istio-proxy"
Oct 30 22:10:24.825104 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:24.824492    4473 cpu_manager.go:414] "containerMap: removing container" podUID="335708bc-083d-4ce1-9158-cc61dcac93da" containerName="istio-proxy"
Oct 30 22:10:24.825104 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:24.824500    4473 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="335708bc-083d-4ce1-9158-cc61dcac93da" containerName="istio-proxy"
Oct 30 22:10:24.825104 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:24.824520    4473 cpu_manager.go:414] "containerMap: removing container" podUID="ef9eca59-6c6f-4523-9046-4fd0611dacef" containerName="istio-proxy"
Oct 30 22:10:24.825104 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:24.824526    4473 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ef9eca59-6c6f-4523-9046-4fd0611dacef" containerName="istio-proxy"
Oct 30 22:10:24.825104 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:24.824554    4473 cpu_manager.go:414] "containerMap: removing container" podUID="335708bc-083d-4ce1-9158-cc61dcac93da" containerName="istio-proxy"
Oct 30 22:10:24.825104 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:24.824561    4473 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="335708bc-083d-4ce1-9158-cc61dcac93da" containerName="istio-proxy"
Oct 30 22:10:24.825104 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:24.824743    4473 memory_manager.go:365] "RemoveStaleState removing state" podUID="48aa5363-fb6c-4cfe-a76d-e8aec041e46f" containerName="istio-proxy"
Oct 30 22:10:24.825104 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:24.824763    4473 memory_manager.go:365] "RemoveStaleState removing state" podUID="335708bc-083d-4ce1-9158-cc61dcac93da" containerName="istio-proxy"
Oct 30 22:10:24.825104 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:24.824773    4473 memory_manager.go:365] "RemoveStaleState removing state" podUID="ef9eca59-6c6f-4523-9046-4fd0611dacef" containerName="istio-proxy"
Oct 30 22:10:24.825104 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:24.824787    4473 memory_manager.go:365] "RemoveStaleState removing state" podUID="335708bc-083d-4ce1-9158-cc61dcac93da" containerName="istio-proxy"
Oct 30 22:10:24.881696 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:24.881669    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-8f1d78db-6cf8f64455-rhrhn"
Oct 30 22:10:24.923554 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:24.922008    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-8f1d78db-6cf8f64455-rhrhn"]
Oct 30 22:10:25.019341 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:24.965106    4473 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-8f1d78db-cc4745b9f-w79rb"]
Oct 30 22:10:25.033989 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:25.032764    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-8f1d78db-dockercfg-sbkjk\""
Oct 30 22:10:25.350051 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:25.045511    4473 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-8f1d78db\""
Oct 30 22:10:25.600194 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:25.600111    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-8f1d78db-cc4745b9f-w79rb"
Oct 30 22:10:25.600973 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:25.600941    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e8260e3f-690e-40cc-80e7-d31d40f65bc6-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-8f1d78db-6cf8f64455-rhrhn\" (UID: \"e8260e3f-690e-40cc-80e7-d31d40f65bc6\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-8f1d78db-6cf8f64455-rhrhn"
Oct 30 22:10:25.601373 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:25.601352    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mtbkg\" (UniqueName: \"kubernetes.io/projected/e8260e3f-690e-40cc-80e7-d31d40f65bc6-kube-api-access-mtbkg\") pod \"payload-pre-processing-e2e-mt-8f1d78db-6cf8f64455-rhrhn\" (UID: \"e8260e3f-690e-40cc-80e7-d31d40f65bc6\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-8f1d78db-6cf8f64455-rhrhn"
Oct 30 22:10:25.620000 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:25.619811    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-8f1d78db-cc4745b9f-w79rb"]
Oct 30 22:10:25.703606 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:25.703569    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mtbkg\" (UniqueName: \"kubernetes.io/projected/e8260e3f-690e-40cc-80e7-d31d40f65bc6-kube-api-access-mtbkg\") pod \"payload-pre-processing-e2e-mt-8f1d78db-6cf8f64455-rhrhn\" (UID: \"e8260e3f-690e-40cc-80e7-d31d40f65bc6\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-8f1d78db-6cf8f64455-rhrhn"
Oct 30 22:10:25.703831 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:25.703809    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e8260e3f-690e-40cc-80e7-d31d40f65bc6-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-8f1d78db-6cf8f64455-rhrhn\" (UID: \"e8260e3f-690e-40cc-80e7-d31d40f65bc6\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-8f1d78db-6cf8f64455-rhrhn"
Oct 30 22:10:25.703882 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:25.703856    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c4fc1f16-aa8a-4d99-b626-fc6738d32fad-plugins-config-volume\") pod \"payload-processing-e2e-mt-8f1d78db-cc4745b9f-w79rb\" (UID: \"c4fc1f16-aa8a-4d99-b626-fc6738d32fad\") " pod="openshift-ingress/payload-processing-e2e-mt-8f1d78db-cc4745b9f-w79rb"
Oct 30 22:10:25.704001 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:25.703978    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x28zc\" (UniqueName: \"kubernetes.io/projected/c4fc1f16-aa8a-4d99-b626-fc6738d32fad-kube-api-access-x28zc\") pod \"payload-processing-e2e-mt-8f1d78db-cc4745b9f-w79rb\" (UID: \"c4fc1f16-aa8a-4d99-b626-fc6738d32fad\") " pod="openshift-ingress/payload-processing-e2e-mt-8f1d78db-cc4745b9f-w79rb"
Oct 30 22:10:25.704546 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:25.704524    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e8260e3f-690e-40cc-80e7-d31d40f65bc6-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-8f1d78db-6cf8f64455-rhrhn\" (UID: \"e8260e3f-690e-40cc-80e7-d31d40f65bc6\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-8f1d78db-6cf8f64455-rhrhn"
Oct 30 22:10:25.797606 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:25.797577    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mtbkg\" (UniqueName: \"kubernetes.io/projected/e8260e3f-690e-40cc-80e7-d31d40f65bc6-kube-api-access-mtbkg\") pod \"payload-pre-processing-e2e-mt-8f1d78db-6cf8f64455-rhrhn\" (UID: \"e8260e3f-690e-40cc-80e7-d31d40f65bc6\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-8f1d78db-6cf8f64455-rhrhn"
Oct 30 22:10:25.805373 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:25.805343    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c4fc1f16-aa8a-4d99-b626-fc6738d32fad-plugins-config-volume\") pod \"payload-processing-e2e-mt-8f1d78db-cc4745b9f-w79rb\" (UID: \"c4fc1f16-aa8a-4d99-b626-fc6738d32fad\") " pod="openshift-ingress/payload-processing-e2e-mt-8f1d78db-cc4745b9f-w79rb"
Oct 30 22:10:25.805373 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:25.805386    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x28zc\" (UniqueName: \"kubernetes.io/projected/c4fc1f16-aa8a-4d99-b626-fc6738d32fad-kube-api-access-x28zc\") pod \"payload-processing-e2e-mt-8f1d78db-cc4745b9f-w79rb\" (UID: \"c4fc1f16-aa8a-4d99-b626-fc6738d32fad\") " pod="openshift-ingress/payload-processing-e2e-mt-8f1d78db-cc4745b9f-w79rb"
Oct 30 22:10:25.806101 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:25.806079    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c4fc1f16-aa8a-4d99-b626-fc6738d32fad-plugins-config-volume\") pod \"payload-processing-e2e-mt-8f1d78db-cc4745b9f-w79rb\" (UID: \"c4fc1f16-aa8a-4d99-b626-fc6738d32fad\") " pod="openshift-ingress/payload-processing-e2e-mt-8f1d78db-cc4745b9f-w79rb"
Oct 30 22:10:25.850737 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:25.850635    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-8f1d78db-6cf8f64455-rhrhn"
Oct 30 22:10:25.867810 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:25.867774    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x28zc\" (UniqueName: \"kubernetes.io/projected/c4fc1f16-aa8a-4d99-b626-fc6738d32fad-kube-api-access-x28zc\") pod \"payload-processing-e2e-mt-8f1d78db-cc4745b9f-w79rb\" (UID: \"c4fc1f16-aa8a-4d99-b626-fc6738d32fad\") " pod="openshift-ingress/payload-processing-e2e-mt-8f1d78db-cc4745b9f-w79rb"
Oct 30 22:10:25.956075 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:25.955000    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-8f1d78db-cc4745b9f-w79rb"
Oct 30 22:10:26.255604 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:26.255574    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-8f1d78db-6cf8f64455-rhrhn"]
Oct 30 22:10:26.259698 ip-10-0-140-18 kubenswrapper[4473]: W1030 22:10:26.259656    4473 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode8260e3f_690e_40cc_80e7_d31d40f65bc6.slice/crio-04379eee640b1b686c24fdb3f7d6d790b41ca94256d75c632cf1ccb6ed164ad1 WatchSource:0}: Error finding container 04379eee640b1b686c24fdb3f7d6d790b41ca94256d75c632cf1ccb6ed164ad1: Status 404 returned error can't find the container with id 04379eee640b1b686c24fdb3f7d6d790b41ca94256d75c632cf1ccb6ed164ad1
Oct 30 22:10:26.376815 ip-10-0-140-18 kubenswrapper[4473]: W1030 22:10:26.376779    4473 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4fc1f16_aa8a_4d99_b626_fc6738d32fad.slice/crio-e661d3671521d395adf6a3c6005cbf1ad50773324dc2d001e794ebbd9ef9e65b WatchSource:0}: Error finding container e661d3671521d395adf6a3c6005cbf1ad50773324dc2d001e794ebbd9ef9e65b: Status 404 returned error can't find the container with id e661d3671521d395adf6a3c6005cbf1ad50773324dc2d001e794ebbd9ef9e65b
Oct 30 22:10:26.377589 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:26.377559    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-8f1d78db-cc4745b9f-w79rb"]
Oct 30 22:10:26.898345 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:26.898313    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-8f1d78db-cc4745b9f-w79rb" event={"ID":"c4fc1f16-aa8a-4d99-b626-fc6738d32fad","Type":"ContainerStarted","Data":"07c13fd494f71bbd5b64bf9d6bb4b3c44282c11eaf72dbf3e74473b44305f127"}
Oct 30 22:10:26.898825 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:26.898367    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-8f1d78db-cc4745b9f-w79rb" event={"ID":"c4fc1f16-aa8a-4d99-b626-fc6738d32fad","Type":"ContainerStarted","Data":"e661d3671521d395adf6a3c6005cbf1ad50773324dc2d001e794ebbd9ef9e65b"}
Oct 30 22:10:26.898825 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:26.898531    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-8f1d78db-cc4745b9f-w79rb"
Oct 30 22:10:26.900313 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:26.900286    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-8f1d78db-6cf8f64455-rhrhn" event={"ID":"e8260e3f-690e-40cc-80e7-d31d40f65bc6","Type":"ContainerStarted","Data":"84eef39479c5a7e62f7af7e6e7ac14087ad1dd4aae81713da7fab2b037716e03"}
Oct 30 22:10:26.900467 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:26.900329    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-8f1d78db-6cf8f64455-rhrhn" event={"ID":"e8260e3f-690e-40cc-80e7-d31d40f65bc6","Type":"ContainerStarted","Data":"04379eee640b1b686c24fdb3f7d6d790b41ca94256d75c632cf1ccb6ed164ad1"}
Oct 30 22:10:26.900467 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:26.900379    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-8f1d78db-6cf8f64455-rhrhn"
Oct 30 22:10:27.017760 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:27.017705    4473 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-8f1d78db-cc4745b9f-w79rb" podStartSLOduration=3.017690952 podStartE2EDuration="3.017690952s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:10:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:10:27.01604738 +0000 UTC m=+1672.789415868" watchObservedRunningTime="2026-10-30 22:10:27.017690952 +0000 UTC m=+1672.791059470"
Oct 30 22:10:27.207990 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:27.201992    4473 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-8f1d78db-6cf8f64455-rhrhn" podStartSLOduration=3.201966644 podStartE2EDuration="3.201966644s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:10:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:10:27.187414739 +0000 UTC m=+1672.960783224" watchObservedRunningTime="2026-10-30 22:10:27.201966644 +0000 UTC m=+1672.975335132"
Oct 30 22:10:27.790234 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:27.790201    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-497924bf-79c65ff565-959fw"]
Oct 30 22:10:27.918205 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:27.918174    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-497924bf-6556b64f5f-wgbj5"]
Oct 30 22:10:30.756947 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:30.752116    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w6-5d36f6-84c6b5685c-zjsct"
Oct 30 22:10:30.767948 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:30.763148    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w6-5d36f6-757f4b4d99-mrs2n"
Oct 30 22:10:32.791060 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:32.790998    4473 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-derive-497924bf-79c65ff565-959fw" podUID="fa9ae053-7674-4271-bb36-522b039a565c" containerName="payload-processing" containerID="cri-o://f3889605b913bca40af0016a027bab08093dd9dcf236df8a9f17b9c3988f542a" gracePeriod=25
Oct 30 22:10:32.918821 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:32.918774    4473 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-derive-497924bf-6556b64f5f-wgbj5" podUID="61628ae3-0dd6-4c03-b96a-12e065362aa0" containerName="payload-pre-processing" containerID="cri-o://74220c8de756621952d8e5fa471e990c7a935fbc00837590a8c66f12a70b0a9b" gracePeriod=25
Oct 30 22:10:33.037732 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:33.037696    4473 generic.go:356] "Generic (PLEG): container finished" podID="fa9ae053-7674-4271-bb36-522b039a565c" containerID="f3889605b913bca40af0016a027bab08093dd9dcf236df8a9f17b9c3988f542a" exitCode=0
Oct 30 22:10:33.037868 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:33.037807    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-497924bf-79c65ff565-959fw" event={"ID":"fa9ae053-7674-4271-bb36-522b039a565c","Type":"ContainerDied","Data":"f3889605b913bca40af0016a027bab08093dd9dcf236df8a9f17b9c3988f542a"}
Oct 30 22:10:33.114907 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:33.114786    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-497924bf-79c65ff565-959fw"
Oct 30 22:10:33.231795 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:33.231763    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-497924bf-6556b64f5f-wgbj5"
Oct 30 22:10:33.262625 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:33.262574    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fa9ae053-7674-4271-bb36-522b039a565c-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fa9ae053-7674-4271-bb36-522b039a565c-plugins-config-volume\") pod \"fa9ae053-7674-4271-bb36-522b039a565c\" (UID: \"fa9ae053-7674-4271-bb36-522b039a565c\") "
Oct 30 22:10:33.262818 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:33.262705    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fa9ae053-7674-4271-bb36-522b039a565c-kube-api-access-gncd7\" (UniqueName: \"kubernetes.io/projected/fa9ae053-7674-4271-bb36-522b039a565c-kube-api-access-gncd7\") pod \"fa9ae053-7674-4271-bb36-522b039a565c\" (UID: \"fa9ae053-7674-4271-bb36-522b039a565c\") "
Oct 30 22:10:33.263024 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:33.263002    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fa9ae053-7674-4271-bb36-522b039a565c-plugins-config-volume" pod "fa9ae053-7674-4271-bb36-522b039a565c" (UID: "fa9ae053-7674-4271-bb36-522b039a565c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:10:33.263704 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:33.263677    4473 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fa9ae053-7674-4271-bb36-522b039a565c-plugins-config-volume\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:10:33.265292 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:33.265262    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fa9ae053-7674-4271-bb36-522b039a565c-kube-api-access-gncd7" pod "fa9ae053-7674-4271-bb36-522b039a565c" (UID: "fa9ae053-7674-4271-bb36-522b039a565c"). InnerVolumeSpecName "kube-api-access-gncd7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:10:33.364810 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:33.364776    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/61628ae3-0dd6-4c03-b96a-12e065362aa0-kube-api-access-7zh98\" (UniqueName: \"kubernetes.io/projected/61628ae3-0dd6-4c03-b96a-12e065362aa0-kube-api-access-7zh98\") pod \"61628ae3-0dd6-4c03-b96a-12e065362aa0\" (UID: \"61628ae3-0dd6-4c03-b96a-12e065362aa0\") "
Oct 30 22:10:33.365225 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:33.365202    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/61628ae3-0dd6-4c03-b96a-12e065362aa0-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/61628ae3-0dd6-4c03-b96a-12e065362aa0-plugins-config-volume\") pod \"61628ae3-0dd6-4c03-b96a-12e065362aa0\" (UID: \"61628ae3-0dd6-4c03-b96a-12e065362aa0\") "
Oct 30 22:10:33.365826 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:33.365796    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/61628ae3-0dd6-4c03-b96a-12e065362aa0-plugins-config-volume" pod "61628ae3-0dd6-4c03-b96a-12e065362aa0" (UID: "61628ae3-0dd6-4c03-b96a-12e065362aa0"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:10:33.366053 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:33.366029    4473 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/61628ae3-0dd6-4c03-b96a-12e065362aa0-plugins-config-volume\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:10:33.366229 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:33.366207    4473 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gncd7\" (UniqueName: \"kubernetes.io/projected/fa9ae053-7674-4271-bb36-522b039a565c-kube-api-access-gncd7\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:10:33.367575 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:33.367548    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/61628ae3-0dd6-4c03-b96a-12e065362aa0-kube-api-access-7zh98" pod "61628ae3-0dd6-4c03-b96a-12e065362aa0" (UID: "61628ae3-0dd6-4c03-b96a-12e065362aa0"). InnerVolumeSpecName "kube-api-access-7zh98". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:10:33.468290 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:33.468261    4473 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7zh98\" (UniqueName: \"kubernetes.io/projected/61628ae3-0dd6-4c03-b96a-12e065362aa0-kube-api-access-7zh98\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:10:34.048935 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:34.048834    4473 generic.go:356] "Generic (PLEG): container finished" podID="f0195818-f111-47ca-a386-005ede718851" containerID="4d00e9f44c1f24b9b0a4c0d1666f42731d1a25290cdb7c1424a51317a0f45cc2" exitCode=0
Oct 30 22:10:34.049466 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:34.048914    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-2a25a4-openshift-default-7bfc77bdbf-9prd7" event={"ID":"f0195818-f111-47ca-a386-005ede718851","Type":"ContainerDied","Data":"4d00e9f44c1f24b9b0a4c0d1666f42731d1a25290cdb7c1424a51317a0f45cc2"}
Oct 30 22:10:34.049466 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:34.049459    4473 scope.go:122] "RemoveContainer" containerID="4d00e9f44c1f24b9b0a4c0d1666f42731d1a25290cdb7c1424a51317a0f45cc2"
Oct 30 22:10:34.050731 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:34.050701    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-497924bf-79c65ff565-959fw"
Oct 30 22:10:34.050731 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:34.050722    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-497924bf-79c65ff565-959fw" event={"ID":"fa9ae053-7674-4271-bb36-522b039a565c","Type":"ContainerDied","Data":"f14ed69a5cd7ea0f6bb9e1d484688e5934c2f29626874eb6067ee316c65e08ae"}
Oct 30 22:10:34.050885 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:34.050778    4473 scope.go:122] "RemoveContainer" containerID="f3889605b913bca40af0016a027bab08093dd9dcf236df8a9f17b9c3988f542a"
Oct 30 22:10:34.051650 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:34.051614    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:10:34.051778 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:34.051719    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:10:34.051844 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:34.051777    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:10:34.052402 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:34.052378    4473 generic.go:356] "Generic (PLEG): container finished" podID="61628ae3-0dd6-4c03-b96a-12e065362aa0" containerID="74220c8de756621952d8e5fa471e990c7a935fbc00837590a8c66f12a70b0a9b" exitCode=0
Oct 30 22:10:34.052695 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:34.052509    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-497924bf-6556b64f5f-wgbj5" event={"ID":"61628ae3-0dd6-4c03-b96a-12e065362aa0","Type":"ContainerDied","Data":"74220c8de756621952d8e5fa471e990c7a935fbc00837590a8c66f12a70b0a9b"}
Oct 30 22:10:34.052695 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:34.052558    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-497924bf-6556b64f5f-wgbj5" event={"ID":"61628ae3-0dd6-4c03-b96a-12e065362aa0","Type":"ContainerDied","Data":"d16f37d415078f5a32481093d33799baab4d439665b779600c499ba03c659165"}
Oct 30 22:10:34.052695 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:34.052599    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-497924bf-6556b64f5f-wgbj5"
Oct 30 22:10:34.070388 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:34.070357    4473 scope.go:122] "RemoveContainer" containerID="74220c8de756621952d8e5fa471e990c7a935fbc00837590a8c66f12a70b0a9b"
Oct 30 22:10:34.094385 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:34.094360    4473 scope.go:122] "RemoveContainer" containerID="74220c8de756621952d8e5fa471e990c7a935fbc00837590a8c66f12a70b0a9b"
Oct 30 22:10:34.095159 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:10:34.095126    4473 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"74220c8de756621952d8e5fa471e990c7a935fbc00837590a8c66f12a70b0a9b\": container with ID starting with 74220c8de756621952d8e5fa471e990c7a935fbc00837590a8c66f12a70b0a9b not found: ID does not exist" containerID="74220c8de756621952d8e5fa471e990c7a935fbc00837590a8c66f12a70b0a9b"
Oct 30 22:10:34.095308 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:34.095171    4473 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"74220c8de756621952d8e5fa471e990c7a935fbc00837590a8c66f12a70b0a9b"} err="failed to get container status \"74220c8de756621952d8e5fa471e990c7a935fbc00837590a8c66f12a70b0a9b\": rpc error: code = NotFound desc = could not find container \"74220c8de756621952d8e5fa471e990c7a935fbc00837590a8c66f12a70b0a9b\": container with ID starting with 74220c8de756621952d8e5fa471e990c7a935fbc00837590a8c66f12a70b0a9b not found: ID does not exist"
Oct 30 22:10:34.389894 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:34.389853    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-497924bf-6556b64f5f-wgbj5"]
Oct 30 22:10:34.391441 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:34.391416    4473 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-497924bf-6556b64f5f-wgbj5"]
Oct 30 22:10:34.495164 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:34.495127    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-497924bf-79c65ff565-959fw"]
Oct 30 22:10:34.503695 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:34.503656    4473 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-497924bf-79c65ff565-959fw"]
Oct 30 22:10:35.067412 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:35.067377    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-2a25a4-openshift-default-7bfc77bdbf-9prd7" event={"ID":"f0195818-f111-47ca-a386-005ede718851","Type":"ContainerStarted","Data":"59c46d799b689368b9bcc9c2c5bff2dd3abcdac78f3aebaa014ba1bd1edf5343"}
Oct 30 22:10:35.345008 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:35.344335    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w2-2a25a4-openshift-default-7bfc77bdbf-9prd7"
Oct 30 22:10:35.350114 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:35.350066    4473 patch_prober.go:29] interesting pod/e2e-shared-a-w2-2a25a4-openshift-default-7bfc77bdbf-9prd7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.107:15021/healthz/ready\": dial tcp 10.133.0.107:15021: connect: connection refused" start-of-body=
Oct 30 22:10:35.350307 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:35.350139    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-2a25a4-openshift-default-7bfc77bdbf-9prd7" podUID="f0195818-f111-47ca-a386-005ede718851" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.107:15021/healthz/ready\": dial tcp 10.133.0.107:15021: connect: connection refused"
Oct 30 22:10:36.347948 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:36.345345    4473 patch_prober.go:29] interesting pod/e2e-shared-a-w2-2a25a4-openshift-default-7bfc77bdbf-9prd7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.107:15021/healthz/ready\": dial tcp 10.133.0.107:15021: connect: connection refused" start-of-body=
Oct 30 22:10:36.347948 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:36.345410    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-2a25a4-openshift-default-7bfc77bdbf-9prd7" podUID="f0195818-f111-47ca-a386-005ede718851" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.107:15021/healthz/ready\": dial tcp 10.133.0.107:15021: connect: connection refused"
Oct 30 22:10:36.364344 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:36.364310    4473 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="61628ae3-0dd6-4c03-b96a-12e065362aa0" path="/var/lib/kubelet/pods/61628ae3-0dd6-4c03-b96a-12e065362aa0/volumes"
Oct 30 22:10:36.364791 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:36.364776    4473 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fa9ae053-7674-4271-bb36-522b039a565c" path="/var/lib/kubelet/pods/fa9ae053-7674-4271-bb36-522b039a565c/volumes"
Oct 30 22:10:37.350388 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:37.350350    4473 patch_prober.go:29] interesting pod/e2e-shared-a-w2-2a25a4-openshift-default-7bfc77bdbf-9prd7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.107:15021/healthz/ready\": dial tcp 10.133.0.107:15021: connect: connection refused" start-of-body=
Oct 30 22:10:37.350829 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:37.350408    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-2a25a4-openshift-default-7bfc77bdbf-9prd7" podUID="f0195818-f111-47ca-a386-005ede718851" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.107:15021/healthz/ready\": dial tcp 10.133.0.107:15021: connect: connection refused"
Oct 30 22:10:37.673096 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:37.673048    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w2-2a25a4-openshift-default-7bfc77bdbf-9prd7"
Oct 30 22:10:37.951193 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:37.950401    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-8f1d78db-6cf8f64455-rhrhn"
Oct 30 22:10:37.952571 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:37.952541    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-8f1d78db-cc4745b9f-w79rb"
Oct 30 22:10:38.364448 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:38.364416    4473 patch_prober.go:29] interesting pod/e2e-shared-a-w2-2a25a4-openshift-default-7bfc77bdbf-9prd7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.107:15021/healthz/ready\": dial tcp 10.133.0.107:15021: connect: connection refused" start-of-body=
Oct 30 22:10:38.365089 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:38.365038    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-2a25a4-openshift-default-7bfc77bdbf-9prd7" podUID="f0195818-f111-47ca-a386-005ede718851" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.107:15021/healthz/ready\": dial tcp 10.133.0.107:15021: connect: connection refused"
Oct 30 22:10:39.350427 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:39.350375    4473 patch_prober.go:29] interesting pod/e2e-shared-a-w2-2a25a4-openshift-default-7bfc77bdbf-9prd7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.107:15021/healthz/ready\": dial tcp 10.133.0.107:15021: connect: connection refused" start-of-body=
Oct 30 22:10:39.350654 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:39.350437    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-2a25a4-openshift-default-7bfc77bdbf-9prd7" podUID="f0195818-f111-47ca-a386-005ede718851" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.107:15021/healthz/ready\": dial tcp 10.133.0.107:15021: connect: connection refused"
Oct 30 22:10:40.242393 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:40.242355    4473 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-242ddaf0-5d8df4c987-fkjbf"]
Oct 30 22:10:40.243365 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:40.243346    4473 cpu_manager.go:414] "containerMap: removing container" podUID="61628ae3-0dd6-4c03-b96a-12e065362aa0" containerName="payload-pre-processing"
Oct 30 22:10:40.243429 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:40.243367    4473 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="61628ae3-0dd6-4c03-b96a-12e065362aa0" containerName="payload-pre-processing"
Oct 30 22:10:40.243429 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:40.243397    4473 cpu_manager.go:414] "containerMap: removing container" podUID="fa9ae053-7674-4271-bb36-522b039a565c" containerName="payload-processing"
Oct 30 22:10:40.243429 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:40.243404    4473 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fa9ae053-7674-4271-bb36-522b039a565c" containerName="payload-processing"
Oct 30 22:10:40.243584 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:40.243573    4473 memory_manager.go:365] "RemoveStaleState removing state" podUID="61628ae3-0dd6-4c03-b96a-12e065362aa0" containerName="payload-pre-processing"
Oct 30 22:10:40.243619 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:40.243594    4473 memory_manager.go:365] "RemoveStaleState removing state" podUID="fa9ae053-7674-4271-bb36-522b039a565c" containerName="payload-processing"
Oct 30 22:10:40.247431 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:40.247408    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-242ddaf0-5d8df4c987-fkjbf"
Oct 30 22:10:40.305473 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:40.305438    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-242ddaf0-5d8df4c987-fkjbf"]
Oct 30 22:10:40.339458 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:40.339419    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mp5zr\" (UniqueName: \"kubernetes.io/projected/d763728d-389e-4e4d-8337-f9ae9d09478f-kube-api-access-mp5zr\") pod \"maas-api-e2e-mt-242ddaf0-5d8df4c987-fkjbf\" (UID: \"d763728d-389e-4e4d-8337-f9ae9d09478f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-242ddaf0-5d8df4c987-fkjbf"
Oct 30 22:10:40.339666 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:40.339572    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d763728d-389e-4e4d-8337-f9ae9d09478f-metrics-certs\") pod \"maas-api-e2e-mt-242ddaf0-5d8df4c987-fkjbf\" (UID: \"d763728d-389e-4e4d-8337-f9ae9d09478f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-242ddaf0-5d8df4c987-fkjbf"
Oct 30 22:10:40.339784 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:40.339764    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d763728d-389e-4e4d-8337-f9ae9d09478f-openshift-service-ca\") pod \"maas-api-e2e-mt-242ddaf0-5d8df4c987-fkjbf\" (UID: \"d763728d-389e-4e4d-8337-f9ae9d09478f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-242ddaf0-5d8df4c987-fkjbf"
Oct 30 22:10:40.340035 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:40.340016    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d763728d-389e-4e4d-8337-f9ae9d09478f-maas-api-tls\") pod \"maas-api-e2e-mt-242ddaf0-5d8df4c987-fkjbf\" (UID: \"d763728d-389e-4e4d-8337-f9ae9d09478f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-242ddaf0-5d8df4c987-fkjbf"
Oct 30 22:10:40.349953 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:40.349894    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-242ddaf0-serving-cert\""
Oct 30 22:10:40.391731 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:40.391673    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w2-2a25a4-openshift-default-7bfc77bdbf-9prd7"
Oct 30 22:10:40.392868 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:40.392837    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w2-2a25a4-openshift-default-7bfc77bdbf-9prd7"
Oct 30 22:10:40.441690 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:40.441643    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mp5zr\" (UniqueName: \"kubernetes.io/projected/d763728d-389e-4e4d-8337-f9ae9d09478f-kube-api-access-mp5zr\") pod \"maas-api-e2e-mt-242ddaf0-5d8df4c987-fkjbf\" (UID: \"d763728d-389e-4e4d-8337-f9ae9d09478f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-242ddaf0-5d8df4c987-fkjbf"
Oct 30 22:10:40.441904 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:40.441718    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d763728d-389e-4e4d-8337-f9ae9d09478f-metrics-certs\") pod \"maas-api-e2e-mt-242ddaf0-5d8df4c987-fkjbf\" (UID: \"d763728d-389e-4e4d-8337-f9ae9d09478f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-242ddaf0-5d8df4c987-fkjbf"
Oct 30 22:10:40.441904 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:40.441821    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d763728d-389e-4e4d-8337-f9ae9d09478f-openshift-service-ca\") pod \"maas-api-e2e-mt-242ddaf0-5d8df4c987-fkjbf\" (UID: \"d763728d-389e-4e4d-8337-f9ae9d09478f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-242ddaf0-5d8df4c987-fkjbf"
Oct 30 22:10:40.442137 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:40.442115    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d763728d-389e-4e4d-8337-f9ae9d09478f-maas-api-tls\") pod \"maas-api-e2e-mt-242ddaf0-5d8df4c987-fkjbf\" (UID: \"d763728d-389e-4e4d-8337-f9ae9d09478f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-242ddaf0-5d8df4c987-fkjbf"
Oct 30 22:10:40.442297 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:10:40.442265    4473 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-242ddaf0-serving-cert: secret "maas-api-e2e-mt-242ddaf0-serving-cert" not found
Oct 30 22:10:40.442403 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:10:40.442339    4473 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d763728d-389e-4e4d-8337-f9ae9d09478f-maas-api-tls podName:d763728d-389e-4e4d-8337-f9ae9d09478f nodeName:}" failed. No retries permitted until 2026-10-30 22:10:40.942320447 +0000 UTC m=+1686.715688930 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/d763728d-389e-4e4d-8337-f9ae9d09478f-maas-api-tls") pod "maas-api-e2e-mt-242ddaf0-5d8df4c987-fkjbf" (UID: "d763728d-389e-4e4d-8337-f9ae9d09478f") : secret "maas-api-e2e-mt-242ddaf0-serving-cert" not found
Oct 30 22:10:40.442636 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:40.442616    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d763728d-389e-4e4d-8337-f9ae9d09478f-openshift-service-ca\") pod \"maas-api-e2e-mt-242ddaf0-5d8df4c987-fkjbf\" (UID: \"d763728d-389e-4e4d-8337-f9ae9d09478f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-242ddaf0-5d8df4c987-fkjbf"
Oct 30 22:10:40.444483 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:40.444462    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d763728d-389e-4e4d-8337-f9ae9d09478f-metrics-certs\") pod \"maas-api-e2e-mt-242ddaf0-5d8df4c987-fkjbf\" (UID: \"d763728d-389e-4e4d-8337-f9ae9d09478f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-242ddaf0-5d8df4c987-fkjbf"
Oct 30 22:10:40.556380 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:40.556299    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mp5zr\" (UniqueName: \"kubernetes.io/projected/d763728d-389e-4e4d-8337-f9ae9d09478f-kube-api-access-mp5zr\") pod \"maas-api-e2e-mt-242ddaf0-5d8df4c987-fkjbf\" (UID: \"d763728d-389e-4e4d-8337-f9ae9d09478f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-242ddaf0-5d8df4c987-fkjbf"
Oct 30 22:10:40.976119 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:40.976083    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d763728d-389e-4e4d-8337-f9ae9d09478f-maas-api-tls\") pod \"maas-api-e2e-mt-242ddaf0-5d8df4c987-fkjbf\" (UID: \"d763728d-389e-4e4d-8337-f9ae9d09478f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-242ddaf0-5d8df4c987-fkjbf"
Oct 30 22:10:40.978871 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:40.978843    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d763728d-389e-4e4d-8337-f9ae9d09478f-maas-api-tls\") pod \"maas-api-e2e-mt-242ddaf0-5d8df4c987-fkjbf\" (UID: \"d763728d-389e-4e4d-8337-f9ae9d09478f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-242ddaf0-5d8df4c987-fkjbf"
Oct 30 22:10:41.170306 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:41.170269    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-242ddaf0-5d8df4c987-fkjbf"
Oct 30 22:10:41.556587 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:41.556562    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-242ddaf0-5d8df4c987-fkjbf"]
Oct 30 22:10:41.557519 ip-10-0-140-18 kubenswrapper[4473]: W1030 22:10:41.557493    4473 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd763728d_389e_4e4d_8337_f9ae9d09478f.slice/crio-86d498560dfd089c6e643ded88f943f1c28e2e490ca2cbb7c405b70b4cd461fd WatchSource:0}: Error finding container 86d498560dfd089c6e643ded88f943f1c28e2e490ca2cbb7c405b70b4cd461fd: Status 404 returned error can't find the container with id 86d498560dfd089c6e643ded88f943f1c28e2e490ca2cbb7c405b70b4cd461fd
Oct 30 22:10:42.138966 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:42.138913    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-242ddaf0-5d8df4c987-fkjbf" event={"ID":"d763728d-389e-4e4d-8337-f9ae9d09478f","Type":"ContainerStarted","Data":"86d498560dfd089c6e643ded88f943f1c28e2e490ca2cbb7c405b70b4cd461fd"}
Oct 30 22:10:43.148332 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:43.148292    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-242ddaf0-5d8df4c987-fkjbf" event={"ID":"d763728d-389e-4e4d-8337-f9ae9d09478f","Type":"ContainerStarted","Data":"fc6c6667ac9d2df46711e4ea6ee3327d3d2c7eadcbe6155844ec132ad78c1f57"}
Oct 30 22:10:43.148769 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:43.148509    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-242ddaf0-5d8df4c987-fkjbf"
Oct 30 22:10:43.268701 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:43.268649    4473 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-242ddaf0-5d8df4c987-fkjbf" podStartSLOduration=2.40408029 podStartE2EDuration="3.268634374s" totalImagesPullingTime="864.554084ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:10:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:10:43.265131488 +0000 UTC m=+1689.038499979" watchObservedRunningTime="2026-10-30 22:10:43.268634374 +0000 UTC m=+1689.042002861"
Oct 30 22:10:49.163134 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:49.163103    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-242ddaf0-5d8df4c987-fkjbf"
Oct 30 22:10:49.811179 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:49.811139    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-90f9e9d3-64b667bb5-hjf95"]
Oct 30 22:10:49.811431 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:49.811407    4473 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-90f9e9d3-64b667bb5-hjf95" podUID="1e012266-60e2-4497-b8b1-9524e23c3421" containerName="maas-api" containerID="cri-o://721f97e99ef368ce4eff984bbce04d94dbf8d541a2b655ddc3f8b5bdcecbf1a7" gracePeriod=30
Oct 30 22:10:50.391093 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:50.390766    4473 generic.go:356] "Generic (PLEG): container finished" podID="1e012266-60e2-4497-b8b1-9524e23c3421" containerID="721f97e99ef368ce4eff984bbce04d94dbf8d541a2b655ddc3f8b5bdcecbf1a7" exitCode=0
Oct 30 22:10:50.477603 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:50.477567    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-90f9e9d3-64b667bb5-hjf95" event={"ID":"1e012266-60e2-4497-b8b1-9524e23c3421","Type":"ContainerDied","Data":"721f97e99ef368ce4eff984bbce04d94dbf8d541a2b655ddc3f8b5bdcecbf1a7"}
Oct 30 22:10:50.500734 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:50.500709    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-90f9e9d3-64b667bb5-hjf95"
Oct 30 22:10:50.509391 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:10:50.509357    4473 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fncvg" containerName="extractor"
Oct 30 22:10:50.650603 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:50.650497    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1e012266-60e2-4497-b8b1-9524e23c3421-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1e012266-60e2-4497-b8b1-9524e23c3421-openshift-service-ca\") pod \"1e012266-60e2-4497-b8b1-9524e23c3421\" (UID: \"1e012266-60e2-4497-b8b1-9524e23c3421\") "
Oct 30 22:10:50.650603 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:50.650557    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1e012266-60e2-4497-b8b1-9524e23c3421-kube-api-access-hz5lk\" (UniqueName: \"kubernetes.io/projected/1e012266-60e2-4497-b8b1-9524e23c3421-kube-api-access-hz5lk\") pod \"1e012266-60e2-4497-b8b1-9524e23c3421\" (UID: \"1e012266-60e2-4497-b8b1-9524e23c3421\") "
Oct 30 22:10:50.650843 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:50.650710    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1e012266-60e2-4497-b8b1-9524e23c3421-metrics-certs\" (UniqueName: \"kubernetes.io/secret/1e012266-60e2-4497-b8b1-9524e23c3421-metrics-certs\") pod \"1e012266-60e2-4497-b8b1-9524e23c3421\" (UID: \"1e012266-60e2-4497-b8b1-9524e23c3421\") "
Oct 30 22:10:50.650843 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:50.650739    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1e012266-60e2-4497-b8b1-9524e23c3421-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1e012266-60e2-4497-b8b1-9524e23c3421-maas-api-tls\") pod \"1e012266-60e2-4497-b8b1-9524e23c3421\" (UID: \"1e012266-60e2-4497-b8b1-9524e23c3421\") "
Oct 30 22:10:50.651001 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:50.650973    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1e012266-60e2-4497-b8b1-9524e23c3421-openshift-service-ca" pod "1e012266-60e2-4497-b8b1-9524e23c3421" (UID: "1e012266-60e2-4497-b8b1-9524e23c3421"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:10:50.651689 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:50.651670    4473 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1e012266-60e2-4497-b8b1-9524e23c3421-openshift-service-ca\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:10:50.653518 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:50.653493    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1e012266-60e2-4497-b8b1-9524e23c3421-metrics-certs" pod "1e012266-60e2-4497-b8b1-9524e23c3421" (UID: "1e012266-60e2-4497-b8b1-9524e23c3421"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:10:50.653604 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:50.653540    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1e012266-60e2-4497-b8b1-9524e23c3421-kube-api-access-hz5lk" pod "1e012266-60e2-4497-b8b1-9524e23c3421" (UID: "1e012266-60e2-4497-b8b1-9524e23c3421"). InnerVolumeSpecName "kube-api-access-hz5lk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:10:50.653684 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:50.653669    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1e012266-60e2-4497-b8b1-9524e23c3421-maas-api-tls" pod "1e012266-60e2-4497-b8b1-9524e23c3421" (UID: "1e012266-60e2-4497-b8b1-9524e23c3421"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:10:50.754292 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:50.754261    4473 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hz5lk\" (UniqueName: \"kubernetes.io/projected/1e012266-60e2-4497-b8b1-9524e23c3421-kube-api-access-hz5lk\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:10:50.754292 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:50.754291    4473 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1e012266-60e2-4497-b8b1-9524e23c3421-metrics-certs\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:10:50.754522 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:50.754303    4473 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1e012266-60e2-4497-b8b1-9524e23c3421-maas-api-tls\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:10:51.399343 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:51.399311    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-90f9e9d3-64b667bb5-hjf95"
Oct 30 22:10:51.399844 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:51.399336    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-90f9e9d3-64b667bb5-hjf95" event={"ID":"1e012266-60e2-4497-b8b1-9524e23c3421","Type":"ContainerDied","Data":"83e2b0888d7435da9a060576ed44f302917fecac23e94efee670525953466a8f"}
Oct 30 22:10:51.399844 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:51.399414    4473 scope.go:122] "RemoveContainer" containerID="721f97e99ef368ce4eff984bbce04d94dbf8d541a2b655ddc3f8b5bdcecbf1a7"
Oct 30 22:10:51.569087 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:51.569049    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-90f9e9d3-64b667bb5-hjf95"]
Oct 30 22:10:51.619663 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:51.619628    4473 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-90f9e9d3-64b667bb5-hjf95"]
Oct 30 22:10:52.121521 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:52.121489    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-90f9e9d3-gw-openshift-default-55569c489-fllqv"]
Oct 30 22:10:52.121733 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:52.121711    4473 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-90f9e9d3-gw-openshift-default-55569c489-fllqv" podUID="bad44c70-cd8b-415c-8d9a-86515b14a266" containerName="istio-proxy" containerID="cri-o://4d7037d25f80726307f2ea28f90ab71e550e3c346bb8ecdf92ca74a9ca9f2c7a" gracePeriod=30
Oct 30 22:10:52.384658 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:52.384563    4473 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1e012266-60e2-4497-b8b1-9524e23c3421" path="/var/lib/kubelet/pods/1e012266-60e2-4497-b8b1-9524e23c3421/volumes"
Oct 30 22:10:53.463714 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:53.463674    4473 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w0-ebddcc-openshift-default-5d98df957-nmwbj"]
Oct 30 22:10:53.464798 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:53.464772    4473 cpu_manager.go:414] "containerMap: removing container" podUID="1e012266-60e2-4497-b8b1-9524e23c3421" containerName="maas-api"
Oct 30 22:10:53.464798 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:53.464798    4473 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1e012266-60e2-4497-b8b1-9524e23c3421" containerName="maas-api"
Oct 30 22:10:53.465085 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:53.465041    4473 memory_manager.go:365] "RemoveStaleState removing state" podUID="1e012266-60e2-4497-b8b1-9524e23c3421" containerName="maas-api"
Oct 30 22:10:53.468964 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:53.468939    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w0-ebddcc-openshift-default-5d98df957-nmwbj"
Oct 30 22:10:53.532947 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:53.528246    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w0-ebddcc-openshift-default-dockercfg-rqr6g\""
Oct 30 22:10:53.620886 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:53.616102    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c15267dd-1a03-40d6-b6b1-ae0dec8cc68f-credential-socket\") pod \"e2e-shared-a-w0-ebddcc-openshift-default-5d98df957-nmwbj\" (UID: \"c15267dd-1a03-40d6-b6b1-ae0dec8cc68f\") " pod="openshift-ingress/e2e-shared-a-w0-ebddcc-openshift-default-5d98df957-nmwbj"
Oct 30 22:10:53.620886 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:53.616275    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c15267dd-1a03-40d6-b6b1-ae0dec8cc68f-istiod-ca-cert\") pod \"e2e-shared-a-w0-ebddcc-openshift-default-5d98df957-nmwbj\" (UID: \"c15267dd-1a03-40d6-b6b1-ae0dec8cc68f\") " pod="openshift-ingress/e2e-shared-a-w0-ebddcc-openshift-default-5d98df957-nmwbj"
Oct 30 22:10:53.620886 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:53.616311    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c15267dd-1a03-40d6-b6b1-ae0dec8cc68f-istio-envoy\") pod \"e2e-shared-a-w0-ebddcc-openshift-default-5d98df957-nmwbj\" (UID: \"c15267dd-1a03-40d6-b6b1-ae0dec8cc68f\") " pod="openshift-ingress/e2e-shared-a-w0-ebddcc-openshift-default-5d98df957-nmwbj"
Oct 30 22:10:53.620886 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:53.616436    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c15267dd-1a03-40d6-b6b1-ae0dec8cc68f-workload-socket\") pod \"e2e-shared-a-w0-ebddcc-openshift-default-5d98df957-nmwbj\" (UID: \"c15267dd-1a03-40d6-b6b1-ae0dec8cc68f\") " pod="openshift-ingress/e2e-shared-a-w0-ebddcc-openshift-default-5d98df957-nmwbj"
Oct 30 22:10:53.620886 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:53.616525    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c15267dd-1a03-40d6-b6b1-ae0dec8cc68f-istio-data\") pod \"e2e-shared-a-w0-ebddcc-openshift-default-5d98df957-nmwbj\" (UID: \"c15267dd-1a03-40d6-b6b1-ae0dec8cc68f\") " pod="openshift-ingress/e2e-shared-a-w0-ebddcc-openshift-default-5d98df957-nmwbj"
Oct 30 22:10:53.620886 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:53.616550    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c15267dd-1a03-40d6-b6b1-ae0dec8cc68f-istio-podinfo\") pod \"e2e-shared-a-w0-ebddcc-openshift-default-5d98df957-nmwbj\" (UID: \"c15267dd-1a03-40d6-b6b1-ae0dec8cc68f\") " pod="openshift-ingress/e2e-shared-a-w0-ebddcc-openshift-default-5d98df957-nmwbj"
Oct 30 22:10:53.620886 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:53.616651    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jnzmz\" (UniqueName: \"kubernetes.io/projected/c15267dd-1a03-40d6-b6b1-ae0dec8cc68f-kube-api-access-jnzmz\") pod \"e2e-shared-a-w0-ebddcc-openshift-default-5d98df957-nmwbj\" (UID: \"c15267dd-1a03-40d6-b6b1-ae0dec8cc68f\") " pod="openshift-ingress/e2e-shared-a-w0-ebddcc-openshift-default-5d98df957-nmwbj"
Oct 30 22:10:53.620886 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:53.616748    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c15267dd-1a03-40d6-b6b1-ae0dec8cc68f-istio-token\") pod \"e2e-shared-a-w0-ebddcc-openshift-default-5d98df957-nmwbj\" (UID: \"c15267dd-1a03-40d6-b6b1-ae0dec8cc68f\") " pod="openshift-ingress/e2e-shared-a-w0-ebddcc-openshift-default-5d98df957-nmwbj"
Oct 30 22:10:53.620886 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:53.616781    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c15267dd-1a03-40d6-b6b1-ae0dec8cc68f-workload-certs\") pod \"e2e-shared-a-w0-ebddcc-openshift-default-5d98df957-nmwbj\" (UID: \"c15267dd-1a03-40d6-b6b1-ae0dec8cc68f\") " pod="openshift-ingress/e2e-shared-a-w0-ebddcc-openshift-default-5d98df957-nmwbj"
Oct 30 22:10:53.853002 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:53.848023    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w0-ebddcc-openshift-default-5d98df957-nmwbj"]
Oct 30 22:10:53.897976 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:53.897942    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c15267dd-1a03-40d6-b6b1-ae0dec8cc68f-istio-data\") pod \"e2e-shared-a-w0-ebddcc-openshift-default-5d98df957-nmwbj\" (UID: \"c15267dd-1a03-40d6-b6b1-ae0dec8cc68f\") " pod="openshift-ingress/e2e-shared-a-w0-ebddcc-openshift-default-5d98df957-nmwbj"
Oct 30 22:10:53.898676 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:53.898652    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c15267dd-1a03-40d6-b6b1-ae0dec8cc68f-istio-podinfo\") pod \"e2e-shared-a-w0-ebddcc-openshift-default-5d98df957-nmwbj\" (UID: \"c15267dd-1a03-40d6-b6b1-ae0dec8cc68f\") " pod="openshift-ingress/e2e-shared-a-w0-ebddcc-openshift-default-5d98df957-nmwbj"
Oct 30 22:10:53.899374 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:53.899354    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jnzmz\" (UniqueName: \"kubernetes.io/projected/c15267dd-1a03-40d6-b6b1-ae0dec8cc68f-kube-api-access-jnzmz\") pod \"e2e-shared-a-w0-ebddcc-openshift-default-5d98df957-nmwbj\" (UID: \"c15267dd-1a03-40d6-b6b1-ae0dec8cc68f\") " pod="openshift-ingress/e2e-shared-a-w0-ebddcc-openshift-default-5d98df957-nmwbj"
Oct 30 22:10:53.899940 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:53.899896    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c15267dd-1a03-40d6-b6b1-ae0dec8cc68f-istio-token\") pod \"e2e-shared-a-w0-ebddcc-openshift-default-5d98df957-nmwbj\" (UID: \"c15267dd-1a03-40d6-b6b1-ae0dec8cc68f\") " pod="openshift-ingress/e2e-shared-a-w0-ebddcc-openshift-default-5d98df957-nmwbj"
Oct 30 22:10:53.900412 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:53.900391    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c15267dd-1a03-40d6-b6b1-ae0dec8cc68f-workload-certs\") pod \"e2e-shared-a-w0-ebddcc-openshift-default-5d98df957-nmwbj\" (UID: \"c15267dd-1a03-40d6-b6b1-ae0dec8cc68f\") " pod="openshift-ingress/e2e-shared-a-w0-ebddcc-openshift-default-5d98df957-nmwbj"
Oct 30 22:10:53.900647 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:53.900631    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c15267dd-1a03-40d6-b6b1-ae0dec8cc68f-credential-socket\") pod \"e2e-shared-a-w0-ebddcc-openshift-default-5d98df957-nmwbj\" (UID: \"c15267dd-1a03-40d6-b6b1-ae0dec8cc68f\") " pod="openshift-ingress/e2e-shared-a-w0-ebddcc-openshift-default-5d98df957-nmwbj"
Oct 30 22:10:53.901507 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:53.901488    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c15267dd-1a03-40d6-b6b1-ae0dec8cc68f-istiod-ca-cert\") pod \"e2e-shared-a-w0-ebddcc-openshift-default-5d98df957-nmwbj\" (UID: \"c15267dd-1a03-40d6-b6b1-ae0dec8cc68f\") " pod="openshift-ingress/e2e-shared-a-w0-ebddcc-openshift-default-5d98df957-nmwbj"
Oct 30 22:10:53.901651 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:53.901637    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c15267dd-1a03-40d6-b6b1-ae0dec8cc68f-istio-envoy\") pod \"e2e-shared-a-w0-ebddcc-openshift-default-5d98df957-nmwbj\" (UID: \"c15267dd-1a03-40d6-b6b1-ae0dec8cc68f\") " pod="openshift-ingress/e2e-shared-a-w0-ebddcc-openshift-default-5d98df957-nmwbj"
Oct 30 22:10:53.901877 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:53.901863    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c15267dd-1a03-40d6-b6b1-ae0dec8cc68f-workload-socket\") pod \"e2e-shared-a-w0-ebddcc-openshift-default-5d98df957-nmwbj\" (UID: \"c15267dd-1a03-40d6-b6b1-ae0dec8cc68f\") " pod="openshift-ingress/e2e-shared-a-w0-ebddcc-openshift-default-5d98df957-nmwbj"
Oct 30 22:10:53.902227 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:53.898583    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c15267dd-1a03-40d6-b6b1-ae0dec8cc68f-istio-data\") pod \"e2e-shared-a-w0-ebddcc-openshift-default-5d98df957-nmwbj\" (UID: \"c15267dd-1a03-40d6-b6b1-ae0dec8cc68f\") " pod="openshift-ingress/e2e-shared-a-w0-ebddcc-openshift-default-5d98df957-nmwbj"
Oct 30 22:10:53.902358 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:53.901016    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c15267dd-1a03-40d6-b6b1-ae0dec8cc68f-workload-certs\") pod \"e2e-shared-a-w0-ebddcc-openshift-default-5d98df957-nmwbj\" (UID: \"c15267dd-1a03-40d6-b6b1-ae0dec8cc68f\") " pod="openshift-ingress/e2e-shared-a-w0-ebddcc-openshift-default-5d98df957-nmwbj"
Oct 30 22:10:53.902789 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:53.902760    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c15267dd-1a03-40d6-b6b1-ae0dec8cc68f-istiod-ca-cert\") pod \"e2e-shared-a-w0-ebddcc-openshift-default-5d98df957-nmwbj\" (UID: \"c15267dd-1a03-40d6-b6b1-ae0dec8cc68f\") " pod="openshift-ingress/e2e-shared-a-w0-ebddcc-openshift-default-5d98df957-nmwbj"
Oct 30 22:10:53.902881 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:53.901236    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c15267dd-1a03-40d6-b6b1-ae0dec8cc68f-credential-socket\") pod \"e2e-shared-a-w0-ebddcc-openshift-default-5d98df957-nmwbj\" (UID: \"c15267dd-1a03-40d6-b6b1-ae0dec8cc68f\") " pod="openshift-ingress/e2e-shared-a-w0-ebddcc-openshift-default-5d98df957-nmwbj"
Oct 30 22:10:53.903116 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:53.903089    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c15267dd-1a03-40d6-b6b1-ae0dec8cc68f-workload-socket\") pod \"e2e-shared-a-w0-ebddcc-openshift-default-5d98df957-nmwbj\" (UID: \"c15267dd-1a03-40d6-b6b1-ae0dec8cc68f\") " pod="openshift-ingress/e2e-shared-a-w0-ebddcc-openshift-default-5d98df957-nmwbj"
Oct 30 22:10:53.910421 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:53.910393    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c15267dd-1a03-40d6-b6b1-ae0dec8cc68f-istio-podinfo\") pod \"e2e-shared-a-w0-ebddcc-openshift-default-5d98df957-nmwbj\" (UID: \"c15267dd-1a03-40d6-b6b1-ae0dec8cc68f\") " pod="openshift-ingress/e2e-shared-a-w0-ebddcc-openshift-default-5d98df957-nmwbj"
Oct 30 22:10:53.928950 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:53.920581    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c15267dd-1a03-40d6-b6b1-ae0dec8cc68f-istio-envoy\") pod \"e2e-shared-a-w0-ebddcc-openshift-default-5d98df957-nmwbj\" (UID: \"c15267dd-1a03-40d6-b6b1-ae0dec8cc68f\") " pod="openshift-ingress/e2e-shared-a-w0-ebddcc-openshift-default-5d98df957-nmwbj"
Oct 30 22:10:53.987271 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:53.985184    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jnzmz\" (UniqueName: \"kubernetes.io/projected/c15267dd-1a03-40d6-b6b1-ae0dec8cc68f-kube-api-access-jnzmz\") pod \"e2e-shared-a-w0-ebddcc-openshift-default-5d98df957-nmwbj\" (UID: \"c15267dd-1a03-40d6-b6b1-ae0dec8cc68f\") " pod="openshift-ingress/e2e-shared-a-w0-ebddcc-openshift-default-5d98df957-nmwbj"
Oct 30 22:10:53.987859 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:53.987831    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c15267dd-1a03-40d6-b6b1-ae0dec8cc68f-istio-token\") pod \"e2e-shared-a-w0-ebddcc-openshift-default-5d98df957-nmwbj\" (UID: \"c15267dd-1a03-40d6-b6b1-ae0dec8cc68f\") " pod="openshift-ingress/e2e-shared-a-w0-ebddcc-openshift-default-5d98df957-nmwbj"
Oct 30 22:10:54.113677 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:54.112282    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w0-ebddcc-openshift-default-5d98df957-nmwbj"
Oct 30 22:10:54.644356 ip-10-0-140-18 kubenswrapper[4473]: W1030 22:10:54.644323    4473 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc15267dd_1a03_40d6_b6b1_ae0dec8cc68f.slice/crio-6282fdf29d19a4135c8c4f29ec1342238a88e6b5d17863cf357cc375256a7885 WatchSource:0}: Error finding container 6282fdf29d19a4135c8c4f29ec1342238a88e6b5d17863cf357cc375256a7885: Status 404 returned error can't find the container with id 6282fdf29d19a4135c8c4f29ec1342238a88e6b5d17863cf357cc375256a7885
Oct 30 22:10:54.645321 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:54.645250    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w0-ebddcc-openshift-default-5d98df957-nmwbj"]
Oct 30 22:10:54.647688 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:54.647649    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:10:54.647778 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:54.647724    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:10:54.647778 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:54.647766    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:10:55.491227 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:55.491184    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w0-ebddcc-openshift-default-5d98df957-nmwbj" event={"ID":"c15267dd-1a03-40d6-b6b1-ae0dec8cc68f","Type":"ContainerStarted","Data":"39290de07fb2eaf514609b7a187a6af002d0fbd2a976ebcda2ba8f7cf3298592"}
Oct 30 22:10:55.491404 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:55.491248    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w0-ebddcc-openshift-default-5d98df957-nmwbj" event={"ID":"c15267dd-1a03-40d6-b6b1-ae0dec8cc68f","Type":"ContainerStarted","Data":"6282fdf29d19a4135c8c4f29ec1342238a88e6b5d17863cf357cc375256a7885"}
Oct 30 22:10:55.627407 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:55.627357    4473 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w0-ebddcc-openshift-default-5d98df957-nmwbj" podStartSLOduration=2.627342396 podStartE2EDuration="2.627342396s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:10:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:10:55.625455261 +0000 UTC m=+1701.398823767" watchObservedRunningTime="2026-10-30 22:10:55.627342396 +0000 UTC m=+1701.400710883"
Oct 30 22:10:56.113240 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:56.113202    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w0-ebddcc-openshift-default-5d98df957-nmwbj"
Oct 30 22:10:56.119723 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:56.119695    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w0-ebddcc-openshift-default-5d98df957-nmwbj"
Oct 30 22:10:56.535998 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:56.531557    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w0-ebddcc-openshift-default-5d98df957-nmwbj"
Oct 30 22:10:56.579604 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:56.579263    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w0-ebddcc-openshift-default-5d98df957-nmwbj"
Oct 30 22:10:57.406184 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:57.406121    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-90f9e9d3-gw-openshift-default-55569c489-fllqv"
Oct 30 22:10:57.487549 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:57.487512    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/bad44c70-cd8b-415c-8d9a-86515b14a266-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bad44c70-cd8b-415c-8d9a-86515b14a266-istiod-ca-cert\") pod \"bad44c70-cd8b-415c-8d9a-86515b14a266\" (UID: \"bad44c70-cd8b-415c-8d9a-86515b14a266\") "
Oct 30 22:10:57.487717 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:57.487616    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bad44c70-cd8b-415c-8d9a-86515b14a266-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bad44c70-cd8b-415c-8d9a-86515b14a266-workload-certs\") pod \"bad44c70-cd8b-415c-8d9a-86515b14a266\" (UID: \"bad44c70-cd8b-415c-8d9a-86515b14a266\") "
Oct 30 22:10:57.487717 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:57.487645    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bad44c70-cd8b-415c-8d9a-86515b14a266-kube-api-access-sdkb6\" (UniqueName: \"kubernetes.io/projected/bad44c70-cd8b-415c-8d9a-86515b14a266-kube-api-access-sdkb6\") pod \"bad44c70-cd8b-415c-8d9a-86515b14a266\" (UID: \"bad44c70-cd8b-415c-8d9a-86515b14a266\") "
Oct 30 22:10:57.487717 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:57.487661    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bad44c70-cd8b-415c-8d9a-86515b14a266-istio-token\" (UniqueName: \"kubernetes.io/projected/bad44c70-cd8b-415c-8d9a-86515b14a266-istio-token\") pod \"bad44c70-cd8b-415c-8d9a-86515b14a266\" (UID: \"bad44c70-cd8b-415c-8d9a-86515b14a266\") "
Oct 30 22:10:57.487869 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:57.487741    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bad44c70-cd8b-415c-8d9a-86515b14a266-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bad44c70-cd8b-415c-8d9a-86515b14a266-workload-socket\") pod \"bad44c70-cd8b-415c-8d9a-86515b14a266\" (UID: \"bad44c70-cd8b-415c-8d9a-86515b14a266\") "
Oct 30 22:10:57.487869 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:57.487820    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bad44c70-cd8b-415c-8d9a-86515b14a266-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bad44c70-cd8b-415c-8d9a-86515b14a266-credential-socket\") pod \"bad44c70-cd8b-415c-8d9a-86515b14a266\" (UID: \"bad44c70-cd8b-415c-8d9a-86515b14a266\") "
Oct 30 22:10:57.487869 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:57.487839    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bad44c70-cd8b-415c-8d9a-86515b14a266-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bad44c70-cd8b-415c-8d9a-86515b14a266-istio-envoy\") pod \"bad44c70-cd8b-415c-8d9a-86515b14a266\" (UID: \"bad44c70-cd8b-415c-8d9a-86515b14a266\") "
Oct 30 22:10:57.488026 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:57.487932    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bad44c70-cd8b-415c-8d9a-86515b14a266-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bad44c70-cd8b-415c-8d9a-86515b14a266-istio-data\") pod \"bad44c70-cd8b-415c-8d9a-86515b14a266\" (UID: \"bad44c70-cd8b-415c-8d9a-86515b14a266\") "
Oct 30 22:10:57.488026 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:57.487967    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/bad44c70-cd8b-415c-8d9a-86515b14a266-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bad44c70-cd8b-415c-8d9a-86515b14a266-istio-podinfo\") pod \"bad44c70-cd8b-415c-8d9a-86515b14a266\" (UID: \"bad44c70-cd8b-415c-8d9a-86515b14a266\") "
Oct 30 22:10:57.488026 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:57.487979    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bad44c70-cd8b-415c-8d9a-86515b14a266-workload-certs" pod "bad44c70-cd8b-415c-8d9a-86515b14a266" (UID: "bad44c70-cd8b-415c-8d9a-86515b14a266"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:10:57.488154 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:57.488045    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bad44c70-cd8b-415c-8d9a-86515b14a266-istiod-ca-cert" pod "bad44c70-cd8b-415c-8d9a-86515b14a266" (UID: "bad44c70-cd8b-415c-8d9a-86515b14a266"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:10:57.488216 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:57.488150    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bad44c70-cd8b-415c-8d9a-86515b14a266-workload-socket" pod "bad44c70-cd8b-415c-8d9a-86515b14a266" (UID: "bad44c70-cd8b-415c-8d9a-86515b14a266"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:10:57.488399 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:57.488305    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bad44c70-cd8b-415c-8d9a-86515b14a266-credential-socket" pod "bad44c70-cd8b-415c-8d9a-86515b14a266" (UID: "bad44c70-cd8b-415c-8d9a-86515b14a266"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:10:57.488399 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:57.488330    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bad44c70-cd8b-415c-8d9a-86515b14a266-istio-data" pod "bad44c70-cd8b-415c-8d9a-86515b14a266" (UID: "bad44c70-cd8b-415c-8d9a-86515b14a266"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:10:57.488787 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:57.488742    4473 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bad44c70-cd8b-415c-8d9a-86515b14a266-workload-certs\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:10:57.488787 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:57.488768    4473 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bad44c70-cd8b-415c-8d9a-86515b14a266-workload-socket\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:10:57.488787 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:57.488781    4473 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bad44c70-cd8b-415c-8d9a-86515b14a266-credential-socket\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:10:57.488994 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:57.488791    4473 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bad44c70-cd8b-415c-8d9a-86515b14a266-istio-data\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:10:57.488994 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:57.488802    4473 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bad44c70-cd8b-415c-8d9a-86515b14a266-istiod-ca-cert\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:10:57.490368 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:57.490346    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bad44c70-cd8b-415c-8d9a-86515b14a266-istio-envoy" pod "bad44c70-cd8b-415c-8d9a-86515b14a266" (UID: "bad44c70-cd8b-415c-8d9a-86515b14a266"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:10:57.490368 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:57.490351    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/bad44c70-cd8b-415c-8d9a-86515b14a266-istio-podinfo" pod "bad44c70-cd8b-415c-8d9a-86515b14a266" (UID: "bad44c70-cd8b-415c-8d9a-86515b14a266"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:10:57.490643 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:57.490623    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bad44c70-cd8b-415c-8d9a-86515b14a266-kube-api-access-sdkb6" pod "bad44c70-cd8b-415c-8d9a-86515b14a266" (UID: "bad44c70-cd8b-415c-8d9a-86515b14a266"). InnerVolumeSpecName "kube-api-access-sdkb6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:10:57.490728 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:57.490662    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bad44c70-cd8b-415c-8d9a-86515b14a266-istio-token" pod "bad44c70-cd8b-415c-8d9a-86515b14a266" (UID: "bad44c70-cd8b-415c-8d9a-86515b14a266"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:10:57.540073 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:57.540033    4473 generic.go:356] "Generic (PLEG): container finished" podID="bad44c70-cd8b-415c-8d9a-86515b14a266" containerID="4d7037d25f80726307f2ea28f90ab71e550e3c346bb8ecdf92ca74a9ca9f2c7a" exitCode=0
Oct 30 22:10:57.540248 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:57.540092    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-90f9e9d3-gw-openshift-default-55569c489-fllqv"
Oct 30 22:10:57.540248 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:57.540123    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-90f9e9d3-gw-openshift-default-55569c489-fllqv" event={"ID":"bad44c70-cd8b-415c-8d9a-86515b14a266","Type":"ContainerDied","Data":"4d7037d25f80726307f2ea28f90ab71e550e3c346bb8ecdf92ca74a9ca9f2c7a"}
Oct 30 22:10:57.540248 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:57.540180    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-90f9e9d3-gw-openshift-default-55569c489-fllqv" event={"ID":"bad44c70-cd8b-415c-8d9a-86515b14a266","Type":"ContainerDied","Data":"d5b2dc412589cf9faaf48e6975e1b1a7f501d91af6cf4962126e2f667718e1ea"}
Oct 30 22:10:57.540248 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:57.540209    4473 scope.go:122] "RemoveContainer" containerID="4d7037d25f80726307f2ea28f90ab71e550e3c346bb8ecdf92ca74a9ca9f2c7a"
Oct 30 22:10:57.552546 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:57.552526    4473 scope.go:122] "RemoveContainer" containerID="67155ca335eefca79662fdd7a204683fc189865ba832b1fbc7ad95336c047da9"
Oct 30 22:10:57.563095 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:57.563065    4473 scope.go:122] "RemoveContainer" containerID="4d7037d25f80726307f2ea28f90ab71e550e3c346bb8ecdf92ca74a9ca9f2c7a"
Oct 30 22:10:57.563432 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:10:57.563405    4473 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4d7037d25f80726307f2ea28f90ab71e550e3c346bb8ecdf92ca74a9ca9f2c7a\": container with ID starting with 4d7037d25f80726307f2ea28f90ab71e550e3c346bb8ecdf92ca74a9ca9f2c7a not found: ID does not exist" containerID="4d7037d25f80726307f2ea28f90ab71e550e3c346bb8ecdf92ca74a9ca9f2c7a"
Oct 30 22:10:57.563517 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:57.563449    4473 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4d7037d25f80726307f2ea28f90ab71e550e3c346bb8ecdf92ca74a9ca9f2c7a"} err="failed to get container status \"4d7037d25f80726307f2ea28f90ab71e550e3c346bb8ecdf92ca74a9ca9f2c7a\": rpc error: code = NotFound desc = could not find container \"4d7037d25f80726307f2ea28f90ab71e550e3c346bb8ecdf92ca74a9ca9f2c7a\": container with ID starting with 4d7037d25f80726307f2ea28f90ab71e550e3c346bb8ecdf92ca74a9ca9f2c7a not found: ID does not exist"
Oct 30 22:10:57.563517 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:57.563479    4473 scope.go:122] "RemoveContainer" containerID="67155ca335eefca79662fdd7a204683fc189865ba832b1fbc7ad95336c047da9"
Oct 30 22:10:57.563794 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:10:57.563775    4473 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"67155ca335eefca79662fdd7a204683fc189865ba832b1fbc7ad95336c047da9\": container with ID starting with 67155ca335eefca79662fdd7a204683fc189865ba832b1fbc7ad95336c047da9 not found: ID does not exist" containerID="67155ca335eefca79662fdd7a204683fc189865ba832b1fbc7ad95336c047da9"
Oct 30 22:10:57.563859 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:57.563805    4473 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"67155ca335eefca79662fdd7a204683fc189865ba832b1fbc7ad95336c047da9"} err="failed to get container status \"67155ca335eefca79662fdd7a204683fc189865ba832b1fbc7ad95336c047da9\": rpc error: code = NotFound desc = could not find container \"67155ca335eefca79662fdd7a204683fc189865ba832b1fbc7ad95336c047da9\": container with ID starting with 67155ca335eefca79662fdd7a204683fc189865ba832b1fbc7ad95336c047da9 not found: ID does not exist"
Oct 30 22:10:57.591490 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:57.591457    4473 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sdkb6\" (UniqueName: \"kubernetes.io/projected/bad44c70-cd8b-415c-8d9a-86515b14a266-kube-api-access-sdkb6\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:10:57.591490 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:57.591489    4473 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bad44c70-cd8b-415c-8d9a-86515b14a266-istio-token\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:10:57.591743 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:57.591504    4473 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bad44c70-cd8b-415c-8d9a-86515b14a266-istio-envoy\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:10:57.591743 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:57.591519    4473 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bad44c70-cd8b-415c-8d9a-86515b14a266-istio-podinfo\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:10:57.749103 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:57.747219    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-90f9e9d3-gw-openshift-default-55569c489-fllqv"]
Oct 30 22:10:57.758055 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:57.756848    4473 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-90f9e9d3-gw-openshift-default-55569c489-fllqv"]
Oct 30 22:10:58.360846 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:58.360807    4473 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="bad44c70-cd8b-415c-8d9a-86515b14a266" path="/var/lib/kubelet/pods/bad44c70-cd8b-415c-8d9a-86515b14a266/volumes"
Oct 30 22:10:58.498560 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:58.498520    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-jvp7n"]
Oct 30 22:10:58.499372 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:58.498790    4473 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-jvp7n" podUID="091485c0-17be-49be-a5bc-a26a9b2e89a3" containerName="istio-proxy" containerID="cri-o://4992977dde23e771b8de76f1efc81470ebec5cc87566438728b5abe889ea2278" gracePeriod=30
Oct 30 22:10:58.539026 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:58.538989    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-mrlnn"]
Oct 30 22:10:58.539212 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:58.539191    4473 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-mrlnn" podUID="198415ba-1770-4667-8703-4693d90c8fe8" containerName="istio-proxy" containerID="cri-o://0516329edc3602ca725ecb3061ec2b3bc02f8407fedd9323c226f0393d0b800f" gracePeriod=30
Oct 30 22:10:58.551403 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:58.551370    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-chw2v"]
Oct 30 22:10:58.551615 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:58.551586    4473 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-chw2v" podUID="65b0ee0a-4761-40f3-bd4d-1b20dc75e233" containerName="istio-proxy" containerID="cri-o://4e96f42d773cab25ab394ce67e08b9e60080a9dab237b8cc30bc7e6d1a0cce25" gracePeriod=30
Oct 30 22:10:58.555851 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:58.555818    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-vncql"]
Oct 30 22:10:58.556090 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:10:58.556069    4473 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-vncql" podUID="06f032d9-4824-42fd-a92f-4cb00284ae32" containerName="istio-proxy" containerID="cri-o://b411d21561d9608a68fa937208577fd21f399d91f5052522426b09803e7d6067" gracePeriod=30
Oct 30 22:11:00.848413 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:00.848382    4473 patch_prober.go:29] interesting pod/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-jvp7n container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 22:11:00.848822 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:00.848433    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-jvp7n" podUID="091485c0-17be-49be-a5bc-a26a9b2e89a3" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 22:11:02.531879 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:02.531797    4473 patch_prober.go:29] interesting pod/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-chw2v container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 22:11:02.532431 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:02.531887    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-chw2v" podUID="65b0ee0a-4761-40f3-bd4d-1b20dc75e233" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 22:11:02.538139 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:02.538105    4473 patch_prober.go:29] interesting pod/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-vncql container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 22:11:02.538305 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:02.538171    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-vncql" podUID="06f032d9-4824-42fd-a92f-4cb00284ae32" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 22:11:03.181981 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:03.181942    4473 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w0-ebddcc-75799c8f95-nqghx"]
Oct 30 22:11:03.195834 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:03.195799    4473 cpu_manager.go:414] "containerMap: removing container" podUID="bad44c70-cd8b-415c-8d9a-86515b14a266" containerName="istio-proxy"
Oct 30 22:11:03.216165 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:03.216130    4473 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bad44c70-cd8b-415c-8d9a-86515b14a266" containerName="istio-proxy"
Oct 30 22:11:03.216901 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:03.216879    4473 cpu_manager.go:414] "containerMap: removing container" podUID="bad44c70-cd8b-415c-8d9a-86515b14a266" containerName="istio-proxy"
Oct 30 22:11:03.217297 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:03.217278    4473 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bad44c70-cd8b-415c-8d9a-86515b14a266" containerName="istio-proxy"
Oct 30 22:11:03.227948 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:03.217700    4473 memory_manager.go:365] "RemoveStaleState removing state" podUID="bad44c70-cd8b-415c-8d9a-86515b14a266" containerName="istio-proxy"
Oct 30 22:11:03.357803 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:03.338209    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-ebddcc-75799c8f95-nqghx"
Oct 30 22:11:03.366431 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:03.366398    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w0-ebddcc-75799c8f95-nqghx"]
Oct 30 22:11:03.412296 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:03.412267    4473 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w0-ebddcc\""
Oct 30 22:11:03.469142 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:03.469058    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w0-ebddcc-dockercfg-vj5b6\""
Oct 30 22:11:03.481132 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:03.481093    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jm9s6\" (UniqueName: \"kubernetes.io/projected/2102f3d2-e20b-402d-81ec-d6b4923df356-kube-api-access-jm9s6\") pod \"payload-pre-processing-e2e-shared-a-w0-ebddcc-75799c8f95-nqghx\" (UID: \"2102f3d2-e20b-402d-81ec-d6b4923df356\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-ebddcc-75799c8f95-nqghx"
Oct 30 22:11:03.504675 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:03.504628    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2102f3d2-e20b-402d-81ec-d6b4923df356-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w0-ebddcc-75799c8f95-nqghx\" (UID: \"2102f3d2-e20b-402d-81ec-d6b4923df356\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-ebddcc-75799c8f95-nqghx"
Oct 30 22:11:03.687309 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:03.670280    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jm9s6\" (UniqueName: \"kubernetes.io/projected/2102f3d2-e20b-402d-81ec-d6b4923df356-kube-api-access-jm9s6\") pod \"payload-pre-processing-e2e-shared-a-w0-ebddcc-75799c8f95-nqghx\" (UID: \"2102f3d2-e20b-402d-81ec-d6b4923df356\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-ebddcc-75799c8f95-nqghx"
Oct 30 22:11:03.687309 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:03.670427    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2102f3d2-e20b-402d-81ec-d6b4923df356-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w0-ebddcc-75799c8f95-nqghx\" (UID: \"2102f3d2-e20b-402d-81ec-d6b4923df356\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-ebddcc-75799c8f95-nqghx"
Oct 30 22:11:03.704105 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:03.704014    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2102f3d2-e20b-402d-81ec-d6b4923df356-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w0-ebddcc-75799c8f95-nqghx\" (UID: \"2102f3d2-e20b-402d-81ec-d6b4923df356\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-ebddcc-75799c8f95-nqghx"
Oct 30 22:11:03.730978 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:03.720127    4473 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w0-ebddcc-5f9dc96795-4z2w2"]
Oct 30 22:11:03.734525 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:03.733404    4473 memory_manager.go:365] "RemoveStaleState removing state" podUID="bad44c70-cd8b-415c-8d9a-86515b14a266" containerName="istio-proxy"
Oct 30 22:11:03.837108 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:03.835240    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w0-ebddcc-5f9dc96795-4z2w2"
Oct 30 22:11:03.907947 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:03.901530    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jm9s6\" (UniqueName: \"kubernetes.io/projected/2102f3d2-e20b-402d-81ec-d6b4923df356-kube-api-access-jm9s6\") pod \"payload-pre-processing-e2e-shared-a-w0-ebddcc-75799c8f95-nqghx\" (UID: \"2102f3d2-e20b-402d-81ec-d6b4923df356\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-ebddcc-75799c8f95-nqghx"
Oct 30 22:11:04.047458 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:04.047361    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l8dgc\" (UniqueName: \"kubernetes.io/projected/e94e5f73-3534-41c8-aa76-38ff540bf455-kube-api-access-l8dgc\") pod \"payload-processing-e2e-shared-a-w0-ebddcc-5f9dc96795-4z2w2\" (UID: \"e94e5f73-3534-41c8-aa76-38ff540bf455\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w0-ebddcc-5f9dc96795-4z2w2"
Oct 30 22:11:04.047953 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:04.047901    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e94e5f73-3534-41c8-aa76-38ff540bf455-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w0-ebddcc-5f9dc96795-4z2w2\" (UID: \"e94e5f73-3534-41c8-aa76-38ff540bf455\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w0-ebddcc-5f9dc96795-4z2w2"
Oct 30 22:11:04.087408 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:04.087375    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w0-ebddcc-5f9dc96795-4z2w2"]
Oct 30 22:11:04.114275 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:04.108564    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-ebddcc-75799c8f95-nqghx"
Oct 30 22:11:04.203442 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:04.198517    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l8dgc\" (UniqueName: \"kubernetes.io/projected/e94e5f73-3534-41c8-aa76-38ff540bf455-kube-api-access-l8dgc\") pod \"payload-processing-e2e-shared-a-w0-ebddcc-5f9dc96795-4z2w2\" (UID: \"e94e5f73-3534-41c8-aa76-38ff540bf455\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w0-ebddcc-5f9dc96795-4z2w2"
Oct 30 22:11:04.203442 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:04.198715    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e94e5f73-3534-41c8-aa76-38ff540bf455-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w0-ebddcc-5f9dc96795-4z2w2\" (UID: \"e94e5f73-3534-41c8-aa76-38ff540bf455\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w0-ebddcc-5f9dc96795-4z2w2"
Oct 30 22:11:04.203442 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:04.199756    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e94e5f73-3534-41c8-aa76-38ff540bf455-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w0-ebddcc-5f9dc96795-4z2w2\" (UID: \"e94e5f73-3534-41c8-aa76-38ff540bf455\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w0-ebddcc-5f9dc96795-4z2w2"
Oct 30 22:11:04.547169 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:04.547068    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l8dgc\" (UniqueName: \"kubernetes.io/projected/e94e5f73-3534-41c8-aa76-38ff540bf455-kube-api-access-l8dgc\") pod \"payload-processing-e2e-shared-a-w0-ebddcc-5f9dc96795-4z2w2\" (UID: \"e94e5f73-3534-41c8-aa76-38ff540bf455\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w0-ebddcc-5f9dc96795-4z2w2"
Oct 30 22:11:04.608315 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:04.608235    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w0-ebddcc-5f9dc96795-4z2w2"
Oct 30 22:11:04.658838 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:04.657874    4473 patch_prober.go:29] interesting pod/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-mrlnn container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.84:15021/healthz/ready\": dial tcp 10.133.0.84:15021: connect: connection refused" start-of-body=
Oct 30 22:11:04.658838 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:04.658013    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-mrlnn" podUID="198415ba-1770-4667-8703-4693d90c8fe8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.84:15021/healthz/ready\": dial tcp 10.133.0.84:15021: connect: connection refused"
Oct 30 22:11:04.742309 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:04.740104    4473 generic.go:356] "Generic (PLEG): container finished" podID="091485c0-17be-49be-a5bc-a26a9b2e89a3" containerID="4992977dde23e771b8de76f1efc81470ebec5cc87566438728b5abe889ea2278" exitCode=0
Oct 30 22:11:04.742309 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:04.740214    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-jvp7n" event={"ID":"091485c0-17be-49be-a5bc-a26a9b2e89a3","Type":"ContainerDied","Data":"4992977dde23e771b8de76f1efc81470ebec5cc87566438728b5abe889ea2278"}
Oct 30 22:11:04.751033 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:04.746887    4473 generic.go:356] "Generic (PLEG): container finished" podID="65b0ee0a-4761-40f3-bd4d-1b20dc75e233" containerID="4e96f42d773cab25ab394ce67e08b9e60080a9dab237b8cc30bc7e6d1a0cce25" exitCode=0
Oct 30 22:11:04.751033 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:04.747093    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-chw2v" event={"ID":"65b0ee0a-4761-40f3-bd4d-1b20dc75e233","Type":"ContainerDied","Data":"4e96f42d773cab25ab394ce67e08b9e60080a9dab237b8cc30bc7e6d1a0cce25"}
Oct 30 22:11:04.751033 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:04.748320    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-vncql"
Oct 30 22:11:04.755941 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:04.753375    4473 generic.go:356] "Generic (PLEG): container finished" podID="06f032d9-4824-42fd-a92f-4cb00284ae32" containerID="b411d21561d9608a68fa937208577fd21f399d91f5052522426b09803e7d6067" exitCode=0
Oct 30 22:11:04.755941 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:04.753494    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-vncql" event={"ID":"06f032d9-4824-42fd-a92f-4cb00284ae32","Type":"ContainerDied","Data":"b411d21561d9608a68fa937208577fd21f399d91f5052522426b09803e7d6067"}
Oct 30 22:11:04.755941 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:04.753546    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-vncql" event={"ID":"06f032d9-4824-42fd-a92f-4cb00284ae32","Type":"ContainerDied","Data":"9626db28aead5d161c54505b8eb500288a30cc40d25708e62f8ac4f952225012"}
Oct 30 22:11:04.755941 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:04.753577    4473 scope.go:122] "RemoveContainer" containerID="b411d21561d9608a68fa937208577fd21f399d91f5052522426b09803e7d6067"
Oct 30 22:11:04.771852 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:04.771366    4473 generic.go:356] "Generic (PLEG): container finished" podID="198415ba-1770-4667-8703-4693d90c8fe8" containerID="0516329edc3602ca725ecb3061ec2b3bc02f8407fedd9323c226f0393d0b800f" exitCode=0
Oct 30 22:11:04.771852 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:04.771589    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-mrlnn" event={"ID":"198415ba-1770-4667-8703-4693d90c8fe8","Type":"ContainerDied","Data":"0516329edc3602ca725ecb3061ec2b3bc02f8407fedd9323c226f0393d0b800f"}
Oct 30 22:11:04.787437 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:04.787290    4473 scope.go:122] "RemoveContainer" containerID="b411d21561d9608a68fa937208577fd21f399d91f5052522426b09803e7d6067"
Oct 30 22:11:04.788116 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:11:04.787992    4473 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b411d21561d9608a68fa937208577fd21f399d91f5052522426b09803e7d6067\": container with ID starting with b411d21561d9608a68fa937208577fd21f399d91f5052522426b09803e7d6067 not found: ID does not exist" containerID="b411d21561d9608a68fa937208577fd21f399d91f5052522426b09803e7d6067"
Oct 30 22:11:04.788116 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:04.788032    4473 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b411d21561d9608a68fa937208577fd21f399d91f5052522426b09803e7d6067"} err="failed to get container status \"b411d21561d9608a68fa937208577fd21f399d91f5052522426b09803e7d6067\": rpc error: code = NotFound desc = could not find container \"b411d21561d9608a68fa937208577fd21f399d91f5052522426b09803e7d6067\": container with ID starting with b411d21561d9608a68fa937208577fd21f399d91f5052522426b09803e7d6067 not found: ID does not exist"
Oct 30 22:11:04.928529 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:04.907538    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/06f032d9-4824-42fd-a92f-4cb00284ae32-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/06f032d9-4824-42fd-a92f-4cb00284ae32-istiod-ca-cert\") pod \"06f032d9-4824-42fd-a92f-4cb00284ae32\" (UID: \"06f032d9-4824-42fd-a92f-4cb00284ae32\") "
Oct 30 22:11:04.928529 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:04.907647    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/06f032d9-4824-42fd-a92f-4cb00284ae32-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/06f032d9-4824-42fd-a92f-4cb00284ae32-istio-data\") pod \"06f032d9-4824-42fd-a92f-4cb00284ae32\" (UID: \"06f032d9-4824-42fd-a92f-4cb00284ae32\") "
Oct 30 22:11:04.928529 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:04.907734    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/06f032d9-4824-42fd-a92f-4cb00284ae32-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/06f032d9-4824-42fd-a92f-4cb00284ae32-credential-socket\") pod \"06f032d9-4824-42fd-a92f-4cb00284ae32\" (UID: \"06f032d9-4824-42fd-a92f-4cb00284ae32\") "
Oct 30 22:11:04.928529 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:04.907854    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/06f032d9-4824-42fd-a92f-4cb00284ae32-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/06f032d9-4824-42fd-a92f-4cb00284ae32-istio-envoy\") pod \"06f032d9-4824-42fd-a92f-4cb00284ae32\" (UID: \"06f032d9-4824-42fd-a92f-4cb00284ae32\") "
Oct 30 22:11:04.928529 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:04.908041    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/06f032d9-4824-42fd-a92f-4cb00284ae32-istio-token\" (UniqueName: \"kubernetes.io/projected/06f032d9-4824-42fd-a92f-4cb00284ae32-istio-token\") pod \"06f032d9-4824-42fd-a92f-4cb00284ae32\" (UID: \"06f032d9-4824-42fd-a92f-4cb00284ae32\") "
Oct 30 22:11:04.928529 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:04.908162    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/06f032d9-4824-42fd-a92f-4cb00284ae32-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/06f032d9-4824-42fd-a92f-4cb00284ae32-workload-socket\") pod \"06f032d9-4824-42fd-a92f-4cb00284ae32\" (UID: \"06f032d9-4824-42fd-a92f-4cb00284ae32\") "
Oct 30 22:11:04.928529 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:04.908195    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/06f032d9-4824-42fd-a92f-4cb00284ae32-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/06f032d9-4824-42fd-a92f-4cb00284ae32-istio-podinfo\") pod \"06f032d9-4824-42fd-a92f-4cb00284ae32\" (UID: \"06f032d9-4824-42fd-a92f-4cb00284ae32\") "
Oct 30 22:11:04.928529 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:04.908325    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/06f032d9-4824-42fd-a92f-4cb00284ae32-kube-api-access-mpgfx\" (UniqueName: \"kubernetes.io/projected/06f032d9-4824-42fd-a92f-4cb00284ae32-kube-api-access-mpgfx\") pod \"06f032d9-4824-42fd-a92f-4cb00284ae32\" (UID: \"06f032d9-4824-42fd-a92f-4cb00284ae32\") "
Oct 30 22:11:04.928529 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:04.908392    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/06f032d9-4824-42fd-a92f-4cb00284ae32-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/06f032d9-4824-42fd-a92f-4cb00284ae32-workload-certs\") pod \"06f032d9-4824-42fd-a92f-4cb00284ae32\" (UID: \"06f032d9-4824-42fd-a92f-4cb00284ae32\") "
Oct 30 22:11:04.939082 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:04.937895    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/06f032d9-4824-42fd-a92f-4cb00284ae32-istio-data" pod "06f032d9-4824-42fd-a92f-4cb00284ae32" (UID: "06f032d9-4824-42fd-a92f-4cb00284ae32"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:11:04.939082 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:04.938302    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/06f032d9-4824-42fd-a92f-4cb00284ae32-istiod-ca-cert" pod "06f032d9-4824-42fd-a92f-4cb00284ae32" (UID: "06f032d9-4824-42fd-a92f-4cb00284ae32"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:11:04.939082 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:04.938517    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/06f032d9-4824-42fd-a92f-4cb00284ae32-credential-socket" pod "06f032d9-4824-42fd-a92f-4cb00284ae32" (UID: "06f032d9-4824-42fd-a92f-4cb00284ae32"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:11:04.939082 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:04.938695    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/06f032d9-4824-42fd-a92f-4cb00284ae32-workload-socket" pod "06f032d9-4824-42fd-a92f-4cb00284ae32" (UID: "06f032d9-4824-42fd-a92f-4cb00284ae32"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:11:04.942705 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:04.942357    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/06f032d9-4824-42fd-a92f-4cb00284ae32-istio-token" pod "06f032d9-4824-42fd-a92f-4cb00284ae32" (UID: "06f032d9-4824-42fd-a92f-4cb00284ae32"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:11:04.947774 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:04.947464    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/06f032d9-4824-42fd-a92f-4cb00284ae32-istio-podinfo" pod "06f032d9-4824-42fd-a92f-4cb00284ae32" (UID: "06f032d9-4824-42fd-a92f-4cb00284ae32"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:11:04.947774 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:04.947757    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/06f032d9-4824-42fd-a92f-4cb00284ae32-kube-api-access-mpgfx" pod "06f032d9-4824-42fd-a92f-4cb00284ae32" (UID: "06f032d9-4824-42fd-a92f-4cb00284ae32"). InnerVolumeSpecName "kube-api-access-mpgfx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:11:04.969709 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:04.969674    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/06f032d9-4824-42fd-a92f-4cb00284ae32-workload-certs" pod "06f032d9-4824-42fd-a92f-4cb00284ae32" (UID: "06f032d9-4824-42fd-a92f-4cb00284ae32"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:11:04.977729 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:04.977338    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/06f032d9-4824-42fd-a92f-4cb00284ae32-istio-envoy" pod "06f032d9-4824-42fd-a92f-4cb00284ae32" (UID: "06f032d9-4824-42fd-a92f-4cb00284ae32"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:11:04.985488 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:04.985141    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-chw2v"
Oct 30 22:11:05.006208 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:05.006106    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-mrlnn"
Oct 30 22:11:05.011534 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:05.010665    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/65b0ee0a-4761-40f3-bd4d-1b20dc75e233-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/65b0ee0a-4761-40f3-bd4d-1b20dc75e233-workload-certs\") pod \"65b0ee0a-4761-40f3-bd4d-1b20dc75e233\" (UID: \"65b0ee0a-4761-40f3-bd4d-1b20dc75e233\") "
Oct 30 22:11:05.011534 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:05.010734    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/65b0ee0a-4761-40f3-bd4d-1b20dc75e233-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/65b0ee0a-4761-40f3-bd4d-1b20dc75e233-istio-podinfo\") pod \"65b0ee0a-4761-40f3-bd4d-1b20dc75e233\" (UID: \"65b0ee0a-4761-40f3-bd4d-1b20dc75e233\") "
Oct 30 22:11:05.011534 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:05.010783    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/65b0ee0a-4761-40f3-bd4d-1b20dc75e233-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/65b0ee0a-4761-40f3-bd4d-1b20dc75e233-workload-socket\") pod \"65b0ee0a-4761-40f3-bd4d-1b20dc75e233\" (UID: \"65b0ee0a-4761-40f3-bd4d-1b20dc75e233\") "
Oct 30 22:11:05.011534 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:05.010859    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/65b0ee0a-4761-40f3-bd4d-1b20dc75e233-istio-token\" (UniqueName: \"kubernetes.io/projected/65b0ee0a-4761-40f3-bd4d-1b20dc75e233-istio-token\") pod \"65b0ee0a-4761-40f3-bd4d-1b20dc75e233\" (UID: \"65b0ee0a-4761-40f3-bd4d-1b20dc75e233\") "
Oct 30 22:11:05.011534 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:05.011026    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/65b0ee0a-4761-40f3-bd4d-1b20dc75e233-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/65b0ee0a-4761-40f3-bd4d-1b20dc75e233-istio-data\") pod \"65b0ee0a-4761-40f3-bd4d-1b20dc75e233\" (UID: \"65b0ee0a-4761-40f3-bd4d-1b20dc75e233\") "
Oct 30 22:11:05.011534 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:05.011053    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/65b0ee0a-4761-40f3-bd4d-1b20dc75e233-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/65b0ee0a-4761-40f3-bd4d-1b20dc75e233-istiod-ca-cert\") pod \"65b0ee0a-4761-40f3-bd4d-1b20dc75e233\" (UID: \"65b0ee0a-4761-40f3-bd4d-1b20dc75e233\") "
Oct 30 22:11:05.011534 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:05.011173    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/65b0ee0a-4761-40f3-bd4d-1b20dc75e233-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/65b0ee0a-4761-40f3-bd4d-1b20dc75e233-istio-envoy\") pod \"65b0ee0a-4761-40f3-bd4d-1b20dc75e233\" (UID: \"65b0ee0a-4761-40f3-bd4d-1b20dc75e233\") "
Oct 30 22:11:05.011534 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:05.011242    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/65b0ee0a-4761-40f3-bd4d-1b20dc75e233-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/65b0ee0a-4761-40f3-bd4d-1b20dc75e233-credential-socket\") pod \"65b0ee0a-4761-40f3-bd4d-1b20dc75e233\" (UID: \"65b0ee0a-4761-40f3-bd4d-1b20dc75e233\") "
Oct 30 22:11:05.011534 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:05.011372    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/65b0ee0a-4761-40f3-bd4d-1b20dc75e233-kube-api-access-s4xtt\" (UniqueName: \"kubernetes.io/projected/65b0ee0a-4761-40f3-bd4d-1b20dc75e233-kube-api-access-s4xtt\") pod \"65b0ee0a-4761-40f3-bd4d-1b20dc75e233\" (UID: \"65b0ee0a-4761-40f3-bd4d-1b20dc75e233\") "
Oct 30 22:11:05.012821 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:05.012438    4473 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/06f032d9-4824-42fd-a92f-4cb00284ae32-istio-token\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:11:05.012821 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:05.012463    4473 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/06f032d9-4824-42fd-a92f-4cb00284ae32-workload-socket\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:11:05.012821 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:05.012479    4473 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/06f032d9-4824-42fd-a92f-4cb00284ae32-istio-podinfo\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:11:05.012821 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:05.012488    4473 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mpgfx\" (UniqueName: \"kubernetes.io/projected/06f032d9-4824-42fd-a92f-4cb00284ae32-kube-api-access-mpgfx\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:11:05.012821 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:05.012498    4473 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/06f032d9-4824-42fd-a92f-4cb00284ae32-workload-certs\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:11:05.012821 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:05.012507    4473 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/06f032d9-4824-42fd-a92f-4cb00284ae32-istiod-ca-cert\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:11:05.012821 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:05.012515    4473 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/06f032d9-4824-42fd-a92f-4cb00284ae32-istio-data\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:11:05.012821 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:05.012526    4473 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/06f032d9-4824-42fd-a92f-4cb00284ae32-credential-socket\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:11:05.012821 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:05.012536    4473 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/06f032d9-4824-42fd-a92f-4cb00284ae32-istio-envoy\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:11:05.013437 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:05.013295    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/65b0ee0a-4761-40f3-bd4d-1b20dc75e233-istiod-ca-cert" pod "65b0ee0a-4761-40f3-bd4d-1b20dc75e233" (UID: "65b0ee0a-4761-40f3-bd4d-1b20dc75e233"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:11:05.013663 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:05.013581    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/65b0ee0a-4761-40f3-bd4d-1b20dc75e233-istio-data" pod "65b0ee0a-4761-40f3-bd4d-1b20dc75e233" (UID: "65b0ee0a-4761-40f3-bd4d-1b20dc75e233"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:11:05.013754 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:05.013703    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/65b0ee0a-4761-40f3-bd4d-1b20dc75e233-workload-certs" pod "65b0ee0a-4761-40f3-bd4d-1b20dc75e233" (UID: "65b0ee0a-4761-40f3-bd4d-1b20dc75e233"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:11:05.014095 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:05.013989    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/65b0ee0a-4761-40f3-bd4d-1b20dc75e233-credential-socket" pod "65b0ee0a-4761-40f3-bd4d-1b20dc75e233" (UID: "65b0ee0a-4761-40f3-bd4d-1b20dc75e233"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:11:05.014192 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:05.014169    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/65b0ee0a-4761-40f3-bd4d-1b20dc75e233-workload-socket" pod "65b0ee0a-4761-40f3-bd4d-1b20dc75e233" (UID: "65b0ee0a-4761-40f3-bd4d-1b20dc75e233"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:11:05.017236 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:05.017199    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/65b0ee0a-4761-40f3-bd4d-1b20dc75e233-istio-token" pod "65b0ee0a-4761-40f3-bd4d-1b20dc75e233" (UID: "65b0ee0a-4761-40f3-bd4d-1b20dc75e233"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:11:05.017714 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:05.017680    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/65b0ee0a-4761-40f3-bd4d-1b20dc75e233-istio-envoy" pod "65b0ee0a-4761-40f3-bd4d-1b20dc75e233" (UID: "65b0ee0a-4761-40f3-bd4d-1b20dc75e233"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:11:05.017806 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:05.017784    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/65b0ee0a-4761-40f3-bd4d-1b20dc75e233-kube-api-access-s4xtt" pod "65b0ee0a-4761-40f3-bd4d-1b20dc75e233" (UID: "65b0ee0a-4761-40f3-bd4d-1b20dc75e233"). InnerVolumeSpecName "kube-api-access-s4xtt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:11:05.021187 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:05.021154    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/65b0ee0a-4761-40f3-bd4d-1b20dc75e233-istio-podinfo" pod "65b0ee0a-4761-40f3-bd4d-1b20dc75e233" (UID: "65b0ee0a-4761-40f3-bd4d-1b20dc75e233"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:11:05.029839 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:05.029807    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-497924bf-openshift-default-85499cb6f-fw82l"]
Oct 30 22:11:05.030102 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:05.030078    4473 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-497924bf-openshift-default-85499cb6f-fw82l" podUID="233e0e6f-3447-4d9c-a443-0fa29a3b4adc" containerName="istio-proxy" containerID="cri-o://6216d1dac58cffb05954f1c0d99be4b05accde204017f6172781c223da299951" gracePeriod=30
Oct 30 22:11:05.115684 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:05.115479    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/198415ba-1770-4667-8703-4693d90c8fe8-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/198415ba-1770-4667-8703-4693d90c8fe8-istiod-ca-cert\") pod \"198415ba-1770-4667-8703-4693d90c8fe8\" (UID: \"198415ba-1770-4667-8703-4693d90c8fe8\") "
Oct 30 22:11:05.116098 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:05.116077    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/198415ba-1770-4667-8703-4693d90c8fe8-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/198415ba-1770-4667-8703-4693d90c8fe8-workload-certs\") pod \"198415ba-1770-4667-8703-4693d90c8fe8\" (UID: \"198415ba-1770-4667-8703-4693d90c8fe8\") "
Oct 30 22:11:05.116235 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:05.116218    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/198415ba-1770-4667-8703-4693d90c8fe8-istio-token\" (UniqueName: \"kubernetes.io/projected/198415ba-1770-4667-8703-4693d90c8fe8-istio-token\") pod \"198415ba-1770-4667-8703-4693d90c8fe8\" (UID: \"198415ba-1770-4667-8703-4693d90c8fe8\") "
Oct 30 22:11:05.116337 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:05.116307    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/198415ba-1770-4667-8703-4693d90c8fe8-istiod-ca-cert" pod "198415ba-1770-4667-8703-4693d90c8fe8" (UID: "198415ba-1770-4667-8703-4693d90c8fe8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:11:05.116432 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:05.116418    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/198415ba-1770-4667-8703-4693d90c8fe8-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/198415ba-1770-4667-8703-4693d90c8fe8-workload-socket\") pod \"198415ba-1770-4667-8703-4693d90c8fe8\" (UID: \"198415ba-1770-4667-8703-4693d90c8fe8\") "
Oct 30 22:11:05.116533 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:05.116512    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/198415ba-1770-4667-8703-4693d90c8fe8-workload-certs" pod "198415ba-1770-4667-8703-4693d90c8fe8" (UID: "198415ba-1770-4667-8703-4693d90c8fe8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:11:05.116627 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:05.116613    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/198415ba-1770-4667-8703-4693d90c8fe8-kube-api-access-g6f74\" (UniqueName: \"kubernetes.io/projected/198415ba-1770-4667-8703-4693d90c8fe8-kube-api-access-g6f74\") pod \"198415ba-1770-4667-8703-4693d90c8fe8\" (UID: \"198415ba-1770-4667-8703-4693d90c8fe8\") "
Oct 30 22:11:05.116865 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:05.116850    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/198415ba-1770-4667-8703-4693d90c8fe8-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/198415ba-1770-4667-8703-4693d90c8fe8-istio-data\") pod \"198415ba-1770-4667-8703-4693d90c8fe8\" (UID: \"198415ba-1770-4667-8703-4693d90c8fe8\") "
Oct 30 22:11:05.128043 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:05.121561    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/198415ba-1770-4667-8703-4693d90c8fe8-workload-socket" pod "198415ba-1770-4667-8703-4693d90c8fe8" (UID: "198415ba-1770-4667-8703-4693d90c8fe8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:11:05.128374 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:05.122962    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/198415ba-1770-4667-8703-4693d90c8fe8-istio-data" pod "198415ba-1770-4667-8703-4693d90c8fe8" (UID: "198415ba-1770-4667-8703-4693d90c8fe8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:11:05.137000 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:05.124426    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/198415ba-1770-4667-8703-4693d90c8fe8-kube-api-access-g6f74" pod "198415ba-1770-4667-8703-4693d90c8fe8" (UID: "198415ba-1770-4667-8703-4693d90c8fe8"). InnerVolumeSpecName "kube-api-access-g6f74". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:11:05.176145 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:05.134109    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/198415ba-1770-4667-8703-4693d90c8fe8-istio-token" pod "198415ba-1770-4667-8703-4693d90c8fe8" (UID: "198415ba-1770-4667-8703-4693d90c8fe8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:11:05.176145 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:05.137624    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/198415ba-1770-4667-8703-4693d90c8fe8-istio-envoy" pod "198415ba-1770-4667-8703-4693d90c8fe8" (UID: "198415ba-1770-4667-8703-4693d90c8fe8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:11:05.176145 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:05.128792    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/198415ba-1770-4667-8703-4693d90c8fe8-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/198415ba-1770-4667-8703-4693d90c8fe8-istio-envoy\") pod \"198415ba-1770-4667-8703-4693d90c8fe8\" (UID: \"198415ba-1770-4667-8703-4693d90c8fe8\") "
Oct 30 22:11:05.176145 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:05.170127    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/198415ba-1770-4667-8703-4693d90c8fe8-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/198415ba-1770-4667-8703-4693d90c8fe8-credential-socket\") pod \"198415ba-1770-4667-8703-4693d90c8fe8\" (UID: \"198415ba-1770-4667-8703-4693d90c8fe8\") "
Oct 30 22:11:05.176145 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:05.170231    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/198415ba-1770-4667-8703-4693d90c8fe8-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/198415ba-1770-4667-8703-4693d90c8fe8-istio-podinfo\") pod \"198415ba-1770-4667-8703-4693d90c8fe8\" (UID: \"198415ba-1770-4667-8703-4693d90c8fe8\") "
Oct 30 22:11:05.176145 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:05.171092    4473 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/198415ba-1770-4667-8703-4693d90c8fe8-istiod-ca-cert\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:11:05.176145 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:05.171119    4473 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/198415ba-1770-4667-8703-4693d90c8fe8-workload-certs\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:11:05.185241 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:05.185201    4473 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/198415ba-1770-4667-8703-4693d90c8fe8-istio-token\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:11:05.230392 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:05.230356    4473 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/198415ba-1770-4667-8703-4693d90c8fe8-workload-socket\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:11:05.244273 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:05.244218    4473 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/65b0ee0a-4761-40f3-bd4d-1b20dc75e233-credential-socket\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:11:05.244511 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:05.244494    4473 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g6f74\" (UniqueName: \"kubernetes.io/projected/198415ba-1770-4667-8703-4693d90c8fe8-kube-api-access-g6f74\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:11:05.244611 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:05.244600    4473 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s4xtt\" (UniqueName: \"kubernetes.io/projected/65b0ee0a-4761-40f3-bd4d-1b20dc75e233-kube-api-access-s4xtt\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:11:05.244691 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:05.244680    4473 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/198415ba-1770-4667-8703-4693d90c8fe8-istio-data\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:11:05.244769 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:05.244757    4473 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/198415ba-1770-4667-8703-4693d90c8fe8-istio-envoy\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:11:05.244853 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:05.244841    4473 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/65b0ee0a-4761-40f3-bd4d-1b20dc75e233-workload-certs\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:11:05.244983 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:05.244964    4473 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/65b0ee0a-4761-40f3-bd4d-1b20dc75e233-istio-podinfo\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:11:05.266946 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:05.262030    4473 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/65b0ee0a-4761-40f3-bd4d-1b20dc75e233-workload-socket\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:11:05.266946 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:05.262137    4473 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/65b0ee0a-4761-40f3-bd4d-1b20dc75e233-istio-token\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:11:05.266946 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:05.262152    4473 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/65b0ee0a-4761-40f3-bd4d-1b20dc75e233-istio-data\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:11:05.266946 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:05.262185    4473 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/65b0ee0a-4761-40f3-bd4d-1b20dc75e233-istiod-ca-cert\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:11:05.266946 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:05.262201    4473 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/65b0ee0a-4761-40f3-bd4d-1b20dc75e233-istio-envoy\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:11:05.293957 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:05.291833    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/198415ba-1770-4667-8703-4693d90c8fe8-credential-socket" pod "198415ba-1770-4667-8703-4693d90c8fe8" (UID: "198415ba-1770-4667-8703-4693d90c8fe8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:11:05.438156 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:05.370673    4473 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/198415ba-1770-4667-8703-4693d90c8fe8-credential-socket\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:11:05.522679 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:05.522212    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-jvp7n"
Oct 30 22:11:05.522679 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:05.522447    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/198415ba-1770-4667-8703-4693d90c8fe8-istio-podinfo" pod "198415ba-1770-4667-8703-4693d90c8fe8" (UID: "198415ba-1770-4667-8703-4693d90c8fe8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:11:05.526109 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:05.526061    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w0-ebddcc-75799c8f95-nqghx"]
Oct 30 22:11:05.588180 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:05.588149    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/091485c0-17be-49be-a5bc-a26a9b2e89a3-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/091485c0-17be-49be-a5bc-a26a9b2e89a3-workload-certs\") pod \"091485c0-17be-49be-a5bc-a26a9b2e89a3\" (UID: \"091485c0-17be-49be-a5bc-a26a9b2e89a3\") "
Oct 30 22:11:05.588333 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:05.588199    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/091485c0-17be-49be-a5bc-a26a9b2e89a3-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/091485c0-17be-49be-a5bc-a26a9b2e89a3-istiod-ca-cert\") pod \"091485c0-17be-49be-a5bc-a26a9b2e89a3\" (UID: \"091485c0-17be-49be-a5bc-a26a9b2e89a3\") "
Oct 30 22:11:05.588333 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:05.588267    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/091485c0-17be-49be-a5bc-a26a9b2e89a3-istio-token\" (UniqueName: \"kubernetes.io/projected/091485c0-17be-49be-a5bc-a26a9b2e89a3-istio-token\") pod \"091485c0-17be-49be-a5bc-a26a9b2e89a3\" (UID: \"091485c0-17be-49be-a5bc-a26a9b2e89a3\") "
Oct 30 22:11:05.588333 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:05.588312    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/091485c0-17be-49be-a5bc-a26a9b2e89a3-kube-api-access-7mh8b\" (UniqueName: \"kubernetes.io/projected/091485c0-17be-49be-a5bc-a26a9b2e89a3-kube-api-access-7mh8b\") pod \"091485c0-17be-49be-a5bc-a26a9b2e89a3\" (UID: \"091485c0-17be-49be-a5bc-a26a9b2e89a3\") "
Oct 30 22:11:05.588492 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:05.588352    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/091485c0-17be-49be-a5bc-a26a9b2e89a3-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/091485c0-17be-49be-a5bc-a26a9b2e89a3-credential-socket\") pod \"091485c0-17be-49be-a5bc-a26a9b2e89a3\" (UID: \"091485c0-17be-49be-a5bc-a26a9b2e89a3\") "
Oct 30 22:11:05.588492 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:05.588438    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/091485c0-17be-49be-a5bc-a26a9b2e89a3-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/091485c0-17be-49be-a5bc-a26a9b2e89a3-istio-data\") pod \"091485c0-17be-49be-a5bc-a26a9b2e89a3\" (UID: \"091485c0-17be-49be-a5bc-a26a9b2e89a3\") "
Oct 30 22:11:05.589093 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:05.588640    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/091485c0-17be-49be-a5bc-a26a9b2e89a3-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/091485c0-17be-49be-a5bc-a26a9b2e89a3-istio-envoy\") pod \"091485c0-17be-49be-a5bc-a26a9b2e89a3\" (UID: \"091485c0-17be-49be-a5bc-a26a9b2e89a3\") "
Oct 30 22:11:05.589093 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:05.588438    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/091485c0-17be-49be-a5bc-a26a9b2e89a3-workload-certs" pod "091485c0-17be-49be-a5bc-a26a9b2e89a3" (UID: "091485c0-17be-49be-a5bc-a26a9b2e89a3"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:11:05.589093 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:05.588673    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/091485c0-17be-49be-a5bc-a26a9b2e89a3-credential-socket" pod "091485c0-17be-49be-a5bc-a26a9b2e89a3" (UID: "091485c0-17be-49be-a5bc-a26a9b2e89a3"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:11:05.589093 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:05.588717    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/091485c0-17be-49be-a5bc-a26a9b2e89a3-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/091485c0-17be-49be-a5bc-a26a9b2e89a3-workload-socket\") pod \"091485c0-17be-49be-a5bc-a26a9b2e89a3\" (UID: \"091485c0-17be-49be-a5bc-a26a9b2e89a3\") "
Oct 30 22:11:05.589093 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:05.588771    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/091485c0-17be-49be-a5bc-a26a9b2e89a3-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/091485c0-17be-49be-a5bc-a26a9b2e89a3-istio-podinfo\") pod \"091485c0-17be-49be-a5bc-a26a9b2e89a3\" (UID: \"091485c0-17be-49be-a5bc-a26a9b2e89a3\") "
Oct 30 22:11:05.589093 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:05.588712    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/091485c0-17be-49be-a5bc-a26a9b2e89a3-istiod-ca-cert" pod "091485c0-17be-49be-a5bc-a26a9b2e89a3" (UID: "091485c0-17be-49be-a5bc-a26a9b2e89a3"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:11:05.589093 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:05.588815    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/091485c0-17be-49be-a5bc-a26a9b2e89a3-istio-data" pod "091485c0-17be-49be-a5bc-a26a9b2e89a3" (UID: "091485c0-17be-49be-a5bc-a26a9b2e89a3"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:11:05.589093 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:05.589048    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/091485c0-17be-49be-a5bc-a26a9b2e89a3-workload-socket" pod "091485c0-17be-49be-a5bc-a26a9b2e89a3" (UID: "091485c0-17be-49be-a5bc-a26a9b2e89a3"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:11:05.589770 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:05.589707    4473 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/091485c0-17be-49be-a5bc-a26a9b2e89a3-credential-socket\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:11:05.589770 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:05.589734    4473 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/091485c0-17be-49be-a5bc-a26a9b2e89a3-istio-data\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:11:05.589770 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:05.589749    4473 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/198415ba-1770-4667-8703-4693d90c8fe8-istio-podinfo\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:11:05.589770 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:05.589763    4473 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/091485c0-17be-49be-a5bc-a26a9b2e89a3-workload-socket\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:11:05.591052 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:05.589777    4473 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/091485c0-17be-49be-a5bc-a26a9b2e89a3-workload-certs\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:11:05.591052 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:05.589790    4473 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/091485c0-17be-49be-a5bc-a26a9b2e89a3-istiod-ca-cert\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:11:05.591344 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:05.591320    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/091485c0-17be-49be-a5bc-a26a9b2e89a3-istio-envoy" pod "091485c0-17be-49be-a5bc-a26a9b2e89a3" (UID: "091485c0-17be-49be-a5bc-a26a9b2e89a3"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:11:05.591418 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:05.591373    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/091485c0-17be-49be-a5bc-a26a9b2e89a3-kube-api-access-7mh8b" pod "091485c0-17be-49be-a5bc-a26a9b2e89a3" (UID: "091485c0-17be-49be-a5bc-a26a9b2e89a3"). InnerVolumeSpecName "kube-api-access-7mh8b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:11:05.591553 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:05.591528    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/091485c0-17be-49be-a5bc-a26a9b2e89a3-istio-podinfo" pod "091485c0-17be-49be-a5bc-a26a9b2e89a3" (UID: "091485c0-17be-49be-a5bc-a26a9b2e89a3"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:11:05.591821 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:05.591801    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/091485c0-17be-49be-a5bc-a26a9b2e89a3-istio-token" pod "091485c0-17be-49be-a5bc-a26a9b2e89a3" (UID: "091485c0-17be-49be-a5bc-a26a9b2e89a3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:11:05.635071 ip-10-0-140-18 kubenswrapper[4473]: W1030 22:11:05.635039    4473 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode94e5f73_3534_41c8_aa76_38ff540bf455.slice/crio-af5a7a84af836932b7143554c2b2f615fea27b237cf31ace9729afaac22f9cf6 WatchSource:0}: Error finding container af5a7a84af836932b7143554c2b2f615fea27b237cf31ace9729afaac22f9cf6: Status 404 returned error can't find the container with id af5a7a84af836932b7143554c2b2f615fea27b237cf31ace9729afaac22f9cf6
Oct 30 22:11:05.637055 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:05.637023    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w0-ebddcc-5f9dc96795-4z2w2"]
Oct 30 22:11:05.691966 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:05.691932    4473 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/091485c0-17be-49be-a5bc-a26a9b2e89a3-istio-envoy\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:11:05.691966 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:05.691965    4473 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/091485c0-17be-49be-a5bc-a26a9b2e89a3-istio-podinfo\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:11:05.692234 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:05.691981    4473 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/091485c0-17be-49be-a5bc-a26a9b2e89a3-istio-token\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:11:05.692234 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:05.691996    4473 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7mh8b\" (UniqueName: \"kubernetes.io/projected/091485c0-17be-49be-a5bc-a26a9b2e89a3-kube-api-access-7mh8b\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:11:05.783585 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:05.783552    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-jvp7n"
Oct 30 22:11:05.783585 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:05.783563    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-jvp7n" event={"ID":"091485c0-17be-49be-a5bc-a26a9b2e89a3","Type":"ContainerDied","Data":"9fc7bdbe9a16b5ddc1a9d226593c8e175b3960949b6c5d985047fc88bbe8fac5"}
Oct 30 22:11:05.784319 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:05.783635    4473 scope.go:122] "RemoveContainer" containerID="4992977dde23e771b8de76f1efc81470ebec5cc87566438728b5abe889ea2278"
Oct 30 22:11:05.798157 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:05.798124    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-chw2v" event={"ID":"65b0ee0a-4761-40f3-bd4d-1b20dc75e233","Type":"ContainerDied","Data":"3f4e911bbc235a413c55011a6d82fb2eab792aa36b807e7044dfad2bbe800203"}
Oct 30 22:11:05.804964 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:05.798324    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-chw2v"
Oct 30 22:11:05.805129 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:05.805049    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-vncql"
Oct 30 22:11:05.807457 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:05.807418    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-ebddcc-75799c8f95-nqghx" event={"ID":"2102f3d2-e20b-402d-81ec-d6b4923df356","Type":"ContainerStarted","Data":"ec19e2dfd41a90a10f6da794402cdf55b1441f505fc77ac00500f3f7320e2dad"}
Oct 30 22:11:05.807607 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:05.807485    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-ebddcc-75799c8f95-nqghx" event={"ID":"2102f3d2-e20b-402d-81ec-d6b4923df356","Type":"ContainerStarted","Data":"c049d994d5866d01bafedf36d425e6547f8024a13f8e0ca059e9239597d34e77"}
Oct 30 22:11:05.807607 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:05.807604    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-ebddcc-75799c8f95-nqghx"
Oct 30 22:11:05.810711 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:05.810691    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-mrlnn"
Oct 30 22:11:05.810991 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:05.810714    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-mrlnn" event={"ID":"198415ba-1770-4667-8703-4693d90c8fe8","Type":"ContainerDied","Data":"620f1e53e16b54c81ffdb4e141d0d665a9fa06c5052b76ec62ae6c1c58372152"}
Oct 30 22:11:05.812347 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:05.812329    4473 scope.go:122] "RemoveContainer" containerID="4e96f42d773cab25ab394ce67e08b9e60080a9dab237b8cc30bc7e6d1a0cce25"
Oct 30 22:11:05.812839 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:05.812812    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w0-ebddcc-5f9dc96795-4z2w2" event={"ID":"e94e5f73-3534-41c8-aa76-38ff540bf455","Type":"ContainerStarted","Data":"0ce341f1a36c5f164ab88157d590fd2a7cfd06f28c3282f617237cb18b7869d6"}
Oct 30 22:11:05.812998 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:05.812870    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w0-ebddcc-5f9dc96795-4z2w2"
Oct 30 22:11:05.812998 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:05.812895    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w0-ebddcc-5f9dc96795-4z2w2" event={"ID":"e94e5f73-3534-41c8-aa76-38ff540bf455","Type":"ContainerStarted","Data":"af5a7a84af836932b7143554c2b2f615fea27b237cf31ace9729afaac22f9cf6"}
Oct 30 22:11:05.825092 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:05.825072    4473 scope.go:122] "RemoveContainer" containerID="0516329edc3602ca725ecb3061ec2b3bc02f8407fedd9323c226f0393d0b800f"
Oct 30 22:11:05.986407 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:05.986269    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-jvp7n"]
Oct 30 22:11:05.993603 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:05.993400    4473 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-jvp7n"]
Oct 30 22:11:06.095171 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:06.095106    4473 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-ebddcc-75799c8f95-nqghx" podStartSLOduration=3.095087602 podStartE2EDuration="3.095087602s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:11:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:11:06.092849605 +0000 UTC m=+1711.866218089" watchObservedRunningTime="2026-10-30 22:11:06.095087602 +0000 UTC m=+1711.868456092"
Oct 30 22:11:06.208575 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:06.208539    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-vncql"]
Oct 30 22:11:06.212526 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:06.212502    4473 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-vncql"]
Oct 30 22:11:06.335756 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:06.335671    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-mrlnn"]
Oct 30 22:11:06.577349 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:06.577312    4473 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="06f032d9-4824-42fd-a92f-4cb00284ae32" path="/var/lib/kubelet/pods/06f032d9-4824-42fd-a92f-4cb00284ae32/volumes"
Oct 30 22:11:06.578463 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:06.578356    4473 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="091485c0-17be-49be-a5bc-a26a9b2e89a3" path="/var/lib/kubelet/pods/091485c0-17be-49be-a5bc-a26a9b2e89a3/volumes"
Oct 30 22:11:06.581426 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:06.581399    4473 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-mrlnn"]
Oct 30 22:11:06.581631 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:06.581616    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-chw2v"]
Oct 30 22:11:06.586355 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:06.586178    4473 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-29cbad-openshift-default-6d74d46b79-chw2v"]
Oct 30 22:11:06.684861 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:06.684797    4473 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w0-ebddcc-5f9dc96795-4z2w2" podStartSLOduration=3.684775419 podStartE2EDuration="3.684775419s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:11:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:11:06.682902625 +0000 UTC m=+1712.456271112" watchObservedRunningTime="2026-10-30 22:11:06.684775419 +0000 UTC m=+1712.458143909"
Oct 30 22:11:06.762020 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:06.761984    4473 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w5-e431fc-openshift-default-6d78d86df7-lhwsl"]
Oct 30 22:11:06.762730 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:06.762716    4473 cpu_manager.go:414] "containerMap: removing container" podUID="06f032d9-4824-42fd-a92f-4cb00284ae32" containerName="istio-proxy"
Oct 30 22:11:06.762769 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:06.762732    4473 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="06f032d9-4824-42fd-a92f-4cb00284ae32" containerName="istio-proxy"
Oct 30 22:11:06.762769 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:06.762765    4473 cpu_manager.go:414] "containerMap: removing container" podUID="65b0ee0a-4761-40f3-bd4d-1b20dc75e233" containerName="istio-proxy"
Oct 30 22:11:06.762856 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:06.762769    4473 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="65b0ee0a-4761-40f3-bd4d-1b20dc75e233" containerName="istio-proxy"
Oct 30 22:11:06.762856 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:06.762777    4473 cpu_manager.go:414] "containerMap: removing container" podUID="091485c0-17be-49be-a5bc-a26a9b2e89a3" containerName="istio-proxy"
Oct 30 22:11:06.762856 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:06.762782    4473 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="091485c0-17be-49be-a5bc-a26a9b2e89a3" containerName="istio-proxy"
Oct 30 22:11:06.762856 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:06.762805    4473 cpu_manager.go:414] "containerMap: removing container" podUID="198415ba-1770-4667-8703-4693d90c8fe8" containerName="istio-proxy"
Oct 30 22:11:06.762856 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:06.762809    4473 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="198415ba-1770-4667-8703-4693d90c8fe8" containerName="istio-proxy"
Oct 30 22:11:06.763070 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:06.762947    4473 memory_manager.go:365] "RemoveStaleState removing state" podUID="65b0ee0a-4761-40f3-bd4d-1b20dc75e233" containerName="istio-proxy"
Oct 30 22:11:06.763070 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:06.762958    4473 memory_manager.go:365] "RemoveStaleState removing state" podUID="091485c0-17be-49be-a5bc-a26a9b2e89a3" containerName="istio-proxy"
Oct 30 22:11:06.763070 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:06.762967    4473 memory_manager.go:365] "RemoveStaleState removing state" podUID="198415ba-1770-4667-8703-4693d90c8fe8" containerName="istio-proxy"
Oct 30 22:11:06.763070 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:06.762976    4473 memory_manager.go:365] "RemoveStaleState removing state" podUID="06f032d9-4824-42fd-a92f-4cb00284ae32" containerName="istio-proxy"
Oct 30 22:11:06.766435 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:06.766415    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w5-e431fc-openshift-default-6d78d86df7-lhwsl"
Oct 30 22:11:06.862141 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:06.849787    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w5-e431fc-openshift-default-dockercfg-v8c2c\""
Oct 30 22:11:06.886023 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:06.885078    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/61782e91-1520-4b83-ad8d-8dd944881b52-istio-data\") pod \"e2e-shared-a-w5-e431fc-openshift-default-6d78d86df7-lhwsl\" (UID: \"61782e91-1520-4b83-ad8d-8dd944881b52\") " pod="openshift-ingress/e2e-shared-a-w5-e431fc-openshift-default-6d78d86df7-lhwsl"
Oct 30 22:11:06.886023 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:06.885137    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/61782e91-1520-4b83-ad8d-8dd944881b52-istio-podinfo\") pod \"e2e-shared-a-w5-e431fc-openshift-default-6d78d86df7-lhwsl\" (UID: \"61782e91-1520-4b83-ad8d-8dd944881b52\") " pod="openshift-ingress/e2e-shared-a-w5-e431fc-openshift-default-6d78d86df7-lhwsl"
Oct 30 22:11:06.886023 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:06.885177    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/61782e91-1520-4b83-ad8d-8dd944881b52-istiod-ca-cert\") pod \"e2e-shared-a-w5-e431fc-openshift-default-6d78d86df7-lhwsl\" (UID: \"61782e91-1520-4b83-ad8d-8dd944881b52\") " pod="openshift-ingress/e2e-shared-a-w5-e431fc-openshift-default-6d78d86df7-lhwsl"
Oct 30 22:11:06.886023 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:06.885358    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/61782e91-1520-4b83-ad8d-8dd944881b52-istio-token\") pod \"e2e-shared-a-w5-e431fc-openshift-default-6d78d86df7-lhwsl\" (UID: \"61782e91-1520-4b83-ad8d-8dd944881b52\") " pod="openshift-ingress/e2e-shared-a-w5-e431fc-openshift-default-6d78d86df7-lhwsl"
Oct 30 22:11:06.886023 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:06.885426    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/61782e91-1520-4b83-ad8d-8dd944881b52-workload-certs\") pod \"e2e-shared-a-w5-e431fc-openshift-default-6d78d86df7-lhwsl\" (UID: \"61782e91-1520-4b83-ad8d-8dd944881b52\") " pod="openshift-ingress/e2e-shared-a-w5-e431fc-openshift-default-6d78d86df7-lhwsl"
Oct 30 22:11:06.886023 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:06.885536    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/61782e91-1520-4b83-ad8d-8dd944881b52-workload-socket\") pod \"e2e-shared-a-w5-e431fc-openshift-default-6d78d86df7-lhwsl\" (UID: \"61782e91-1520-4b83-ad8d-8dd944881b52\") " pod="openshift-ingress/e2e-shared-a-w5-e431fc-openshift-default-6d78d86df7-lhwsl"
Oct 30 22:11:06.886023 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:06.885622    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/61782e91-1520-4b83-ad8d-8dd944881b52-istio-envoy\") pod \"e2e-shared-a-w5-e431fc-openshift-default-6d78d86df7-lhwsl\" (UID: \"61782e91-1520-4b83-ad8d-8dd944881b52\") " pod="openshift-ingress/e2e-shared-a-w5-e431fc-openshift-default-6d78d86df7-lhwsl"
Oct 30 22:11:06.886023 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:06.885667    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zlv77\" (UniqueName: \"kubernetes.io/projected/61782e91-1520-4b83-ad8d-8dd944881b52-kube-api-access-zlv77\") pod \"e2e-shared-a-w5-e431fc-openshift-default-6d78d86df7-lhwsl\" (UID: \"61782e91-1520-4b83-ad8d-8dd944881b52\") " pod="openshift-ingress/e2e-shared-a-w5-e431fc-openshift-default-6d78d86df7-lhwsl"
Oct 30 22:11:06.886023 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:06.885742    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/61782e91-1520-4b83-ad8d-8dd944881b52-credential-socket\") pod \"e2e-shared-a-w5-e431fc-openshift-default-6d78d86df7-lhwsl\" (UID: \"61782e91-1520-4b83-ad8d-8dd944881b52\") " pod="openshift-ingress/e2e-shared-a-w5-e431fc-openshift-default-6d78d86df7-lhwsl"
Oct 30 22:11:06.931443 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:06.931384    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w5-e431fc-openshift-default-6d78d86df7-lhwsl"]
Oct 30 22:11:06.994947 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:06.989184    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/61782e91-1520-4b83-ad8d-8dd944881b52-istio-token\") pod \"e2e-shared-a-w5-e431fc-openshift-default-6d78d86df7-lhwsl\" (UID: \"61782e91-1520-4b83-ad8d-8dd944881b52\") " pod="openshift-ingress/e2e-shared-a-w5-e431fc-openshift-default-6d78d86df7-lhwsl"
Oct 30 22:11:06.994947 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:06.989273    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/61782e91-1520-4b83-ad8d-8dd944881b52-workload-certs\") pod \"e2e-shared-a-w5-e431fc-openshift-default-6d78d86df7-lhwsl\" (UID: \"61782e91-1520-4b83-ad8d-8dd944881b52\") " pod="openshift-ingress/e2e-shared-a-w5-e431fc-openshift-default-6d78d86df7-lhwsl"
Oct 30 22:11:06.994947 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:06.989385    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/61782e91-1520-4b83-ad8d-8dd944881b52-workload-socket\") pod \"e2e-shared-a-w5-e431fc-openshift-default-6d78d86df7-lhwsl\" (UID: \"61782e91-1520-4b83-ad8d-8dd944881b52\") " pod="openshift-ingress/e2e-shared-a-w5-e431fc-openshift-default-6d78d86df7-lhwsl"
Oct 30 22:11:06.994947 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:06.989469    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/61782e91-1520-4b83-ad8d-8dd944881b52-istio-envoy\") pod \"e2e-shared-a-w5-e431fc-openshift-default-6d78d86df7-lhwsl\" (UID: \"61782e91-1520-4b83-ad8d-8dd944881b52\") " pod="openshift-ingress/e2e-shared-a-w5-e431fc-openshift-default-6d78d86df7-lhwsl"
Oct 30 22:11:06.994947 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:06.989509    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zlv77\" (UniqueName: \"kubernetes.io/projected/61782e91-1520-4b83-ad8d-8dd944881b52-kube-api-access-zlv77\") pod \"e2e-shared-a-w5-e431fc-openshift-default-6d78d86df7-lhwsl\" (UID: \"61782e91-1520-4b83-ad8d-8dd944881b52\") " pod="openshift-ingress/e2e-shared-a-w5-e431fc-openshift-default-6d78d86df7-lhwsl"
Oct 30 22:11:06.994947 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:06.989580    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/61782e91-1520-4b83-ad8d-8dd944881b52-credential-socket\") pod \"e2e-shared-a-w5-e431fc-openshift-default-6d78d86df7-lhwsl\" (UID: \"61782e91-1520-4b83-ad8d-8dd944881b52\") " pod="openshift-ingress/e2e-shared-a-w5-e431fc-openshift-default-6d78d86df7-lhwsl"
Oct 30 22:11:06.994947 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:06.989644    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/61782e91-1520-4b83-ad8d-8dd944881b52-istio-data\") pod \"e2e-shared-a-w5-e431fc-openshift-default-6d78d86df7-lhwsl\" (UID: \"61782e91-1520-4b83-ad8d-8dd944881b52\") " pod="openshift-ingress/e2e-shared-a-w5-e431fc-openshift-default-6d78d86df7-lhwsl"
Oct 30 22:11:06.994947 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:06.989670    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/61782e91-1520-4b83-ad8d-8dd944881b52-istio-podinfo\") pod \"e2e-shared-a-w5-e431fc-openshift-default-6d78d86df7-lhwsl\" (UID: \"61782e91-1520-4b83-ad8d-8dd944881b52\") " pod="openshift-ingress/e2e-shared-a-w5-e431fc-openshift-default-6d78d86df7-lhwsl"
Oct 30 22:11:06.994947 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:06.989706    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/61782e91-1520-4b83-ad8d-8dd944881b52-istiod-ca-cert\") pod \"e2e-shared-a-w5-e431fc-openshift-default-6d78d86df7-lhwsl\" (UID: \"61782e91-1520-4b83-ad8d-8dd944881b52\") " pod="openshift-ingress/e2e-shared-a-w5-e431fc-openshift-default-6d78d86df7-lhwsl"
Oct 30 22:11:06.994947 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:06.991998    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/61782e91-1520-4b83-ad8d-8dd944881b52-istiod-ca-cert\") pod \"e2e-shared-a-w5-e431fc-openshift-default-6d78d86df7-lhwsl\" (UID: \"61782e91-1520-4b83-ad8d-8dd944881b52\") " pod="openshift-ingress/e2e-shared-a-w5-e431fc-openshift-default-6d78d86df7-lhwsl"
Oct 30 22:11:06.994947 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:06.992619    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/61782e91-1520-4b83-ad8d-8dd944881b52-workload-certs\") pod \"e2e-shared-a-w5-e431fc-openshift-default-6d78d86df7-lhwsl\" (UID: \"61782e91-1520-4b83-ad8d-8dd944881b52\") " pod="openshift-ingress/e2e-shared-a-w5-e431fc-openshift-default-6d78d86df7-lhwsl"
Oct 30 22:11:06.994947 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:06.992908    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/61782e91-1520-4b83-ad8d-8dd944881b52-workload-socket\") pod \"e2e-shared-a-w5-e431fc-openshift-default-6d78d86df7-lhwsl\" (UID: \"61782e91-1520-4b83-ad8d-8dd944881b52\") " pod="openshift-ingress/e2e-shared-a-w5-e431fc-openshift-default-6d78d86df7-lhwsl"
Oct 30 22:11:06.994947 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:06.994556    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/61782e91-1520-4b83-ad8d-8dd944881b52-istio-data\") pod \"e2e-shared-a-w5-e431fc-openshift-default-6d78d86df7-lhwsl\" (UID: \"61782e91-1520-4b83-ad8d-8dd944881b52\") " pod="openshift-ingress/e2e-shared-a-w5-e431fc-openshift-default-6d78d86df7-lhwsl"
Oct 30 22:11:06.995731 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:06.995089    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/61782e91-1520-4b83-ad8d-8dd944881b52-credential-socket\") pod \"e2e-shared-a-w5-e431fc-openshift-default-6d78d86df7-lhwsl\" (UID: \"61782e91-1520-4b83-ad8d-8dd944881b52\") " pod="openshift-ingress/e2e-shared-a-w5-e431fc-openshift-default-6d78d86df7-lhwsl"
Oct 30 22:11:07.025083 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:07.025007    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/61782e91-1520-4b83-ad8d-8dd944881b52-istio-podinfo\") pod \"e2e-shared-a-w5-e431fc-openshift-default-6d78d86df7-lhwsl\" (UID: \"61782e91-1520-4b83-ad8d-8dd944881b52\") " pod="openshift-ingress/e2e-shared-a-w5-e431fc-openshift-default-6d78d86df7-lhwsl"
Oct 30 22:11:07.048995 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:07.045031    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/61782e91-1520-4b83-ad8d-8dd944881b52-istio-envoy\") pod \"e2e-shared-a-w5-e431fc-openshift-default-6d78d86df7-lhwsl\" (UID: \"61782e91-1520-4b83-ad8d-8dd944881b52\") " pod="openshift-ingress/e2e-shared-a-w5-e431fc-openshift-default-6d78d86df7-lhwsl"
Oct 30 22:11:07.077466 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:07.077394    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zlv77\" (UniqueName: \"kubernetes.io/projected/61782e91-1520-4b83-ad8d-8dd944881b52-kube-api-access-zlv77\") pod \"e2e-shared-a-w5-e431fc-openshift-default-6d78d86df7-lhwsl\" (UID: \"61782e91-1520-4b83-ad8d-8dd944881b52\") " pod="openshift-ingress/e2e-shared-a-w5-e431fc-openshift-default-6d78d86df7-lhwsl"
Oct 30 22:11:07.148849 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:07.147537    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/61782e91-1520-4b83-ad8d-8dd944881b52-istio-token\") pod \"e2e-shared-a-w5-e431fc-openshift-default-6d78d86df7-lhwsl\" (UID: \"61782e91-1520-4b83-ad8d-8dd944881b52\") " pod="openshift-ingress/e2e-shared-a-w5-e431fc-openshift-default-6d78d86df7-lhwsl"
Oct 30 22:11:07.383214 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:07.383177    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w5-e431fc-openshift-default-6d78d86df7-lhwsl"
Oct 30 22:11:07.822201 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:07.819852    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w5-e431fc-openshift-default-6d78d86df7-lhwsl"]
Oct 30 22:11:07.822701 ip-10-0-140-18 kubenswrapper[4473]: W1030 22:11:07.822668    4473 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod61782e91_1520_4b83_ad8d_8dd944881b52.slice/crio-243063e8918ace27f3ed559bbf20257884c012df1d5767c346d129949d67e2fa WatchSource:0}: Error finding container 243063e8918ace27f3ed559bbf20257884c012df1d5767c346d129949d67e2fa: Status 404 returned error can't find the container with id 243063e8918ace27f3ed559bbf20257884c012df1d5767c346d129949d67e2fa
Oct 30 22:11:07.825809 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:07.825769    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:11:07.825966 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:07.825864    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:11:07.825966 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:07.825914    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:11:07.964159 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:07.964118    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w5-e431fc-openshift-default-6d78d86df7-lhwsl" event={"ID":"61782e91-1520-4b83-ad8d-8dd944881b52","Type":"ContainerStarted","Data":"414face9ec799d471c949aa7a6ab3c8d0c101d52e8254a13d48e49b791e7069e"}
Oct 30 22:11:07.964649 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:07.964186    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w5-e431fc-openshift-default-6d78d86df7-lhwsl" event={"ID":"61782e91-1520-4b83-ad8d-8dd944881b52","Type":"ContainerStarted","Data":"243063e8918ace27f3ed559bbf20257884c012df1d5767c346d129949d67e2fa"}
Oct 30 22:11:08.090298 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:08.090188    4473 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w5-e431fc-openshift-default-6d78d86df7-lhwsl" podStartSLOduration=2.090172434 podStartE2EDuration="2.090172434s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:11:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:11:08.086169825 +0000 UTC m=+1713.859538313" watchObservedRunningTime="2026-10-30 22:11:08.090172434 +0000 UTC m=+1713.863540920"
Oct 30 22:11:08.360000 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:08.359860    4473 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="198415ba-1770-4667-8703-4693d90c8fe8" path="/var/lib/kubelet/pods/198415ba-1770-4667-8703-4693d90c8fe8/volumes"
Oct 30 22:11:08.360495 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:08.360474    4473 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="65b0ee0a-4761-40f3-bd4d-1b20dc75e233" path="/var/lib/kubelet/pods/65b0ee0a-4761-40f3-bd4d-1b20dc75e233/volumes"
Oct 30 22:11:08.384015 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:08.383976    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w5-e431fc-openshift-default-6d78d86df7-lhwsl"
Oct 30 22:11:08.390519 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:08.390494    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w5-e431fc-openshift-default-6d78d86df7-lhwsl"
Oct 30 22:11:08.973450 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:08.973416    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w5-e431fc-openshift-default-6d78d86df7-lhwsl"
Oct 30 22:11:08.974431 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:08.974410    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w5-e431fc-openshift-default-6d78d86df7-lhwsl"
Oct 30 22:11:10.326703 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:10.326678    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-497924bf-openshift-default-85499cb6f-fw82l"
Oct 30 22:11:10.375427 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:10.375396    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/233e0e6f-3447-4d9c-a443-0fa29a3b4adc-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/233e0e6f-3447-4d9c-a443-0fa29a3b4adc-istiod-ca-cert\") pod \"233e0e6f-3447-4d9c-a443-0fa29a3b4adc\" (UID: \"233e0e6f-3447-4d9c-a443-0fa29a3b4adc\") "
Oct 30 22:11:10.376374 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:10.375451    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/233e0e6f-3447-4d9c-a443-0fa29a3b4adc-kube-api-access-t6sr7\" (UniqueName: \"kubernetes.io/projected/233e0e6f-3447-4d9c-a443-0fa29a3b4adc-kube-api-access-t6sr7\") pod \"233e0e6f-3447-4d9c-a443-0fa29a3b4adc\" (UID: \"233e0e6f-3447-4d9c-a443-0fa29a3b4adc\") "
Oct 30 22:11:10.376374 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:10.375575    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/233e0e6f-3447-4d9c-a443-0fa29a3b4adc-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/233e0e6f-3447-4d9c-a443-0fa29a3b4adc-credential-socket\") pod \"233e0e6f-3447-4d9c-a443-0fa29a3b4adc\" (UID: \"233e0e6f-3447-4d9c-a443-0fa29a3b4adc\") "
Oct 30 22:11:10.376374 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:10.375622    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/233e0e6f-3447-4d9c-a443-0fa29a3b4adc-istio-token\" (UniqueName: \"kubernetes.io/projected/233e0e6f-3447-4d9c-a443-0fa29a3b4adc-istio-token\") pod \"233e0e6f-3447-4d9c-a443-0fa29a3b4adc\" (UID: \"233e0e6f-3447-4d9c-a443-0fa29a3b4adc\") "
Oct 30 22:11:10.376374 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:10.375657    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/233e0e6f-3447-4d9c-a443-0fa29a3b4adc-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/233e0e6f-3447-4d9c-a443-0fa29a3b4adc-istio-podinfo\") pod \"233e0e6f-3447-4d9c-a443-0fa29a3b4adc\" (UID: \"233e0e6f-3447-4d9c-a443-0fa29a3b4adc\") "
Oct 30 22:11:10.376374 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:10.375757    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/233e0e6f-3447-4d9c-a443-0fa29a3b4adc-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/233e0e6f-3447-4d9c-a443-0fa29a3b4adc-istio-envoy\") pod \"233e0e6f-3447-4d9c-a443-0fa29a3b4adc\" (UID: \"233e0e6f-3447-4d9c-a443-0fa29a3b4adc\") "
Oct 30 22:11:10.376374 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:10.375889    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/233e0e6f-3447-4d9c-a443-0fa29a3b4adc-credential-socket" pod "233e0e6f-3447-4d9c-a443-0fa29a3b4adc" (UID: "233e0e6f-3447-4d9c-a443-0fa29a3b4adc"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:11:10.376374 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:10.375976    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/233e0e6f-3447-4d9c-a443-0fa29a3b4adc-istiod-ca-cert" pod "233e0e6f-3447-4d9c-a443-0fa29a3b4adc" (UID: "233e0e6f-3447-4d9c-a443-0fa29a3b4adc"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:11:10.376374 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:10.376036    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/233e0e6f-3447-4d9c-a443-0fa29a3b4adc-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/233e0e6f-3447-4d9c-a443-0fa29a3b4adc-istio-data\") pod \"233e0e6f-3447-4d9c-a443-0fa29a3b4adc\" (UID: \"233e0e6f-3447-4d9c-a443-0fa29a3b4adc\") "
Oct 30 22:11:10.376374 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:10.376120    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/233e0e6f-3447-4d9c-a443-0fa29a3b4adc-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/233e0e6f-3447-4d9c-a443-0fa29a3b4adc-workload-socket\") pod \"233e0e6f-3447-4d9c-a443-0fa29a3b4adc\" (UID: \"233e0e6f-3447-4d9c-a443-0fa29a3b4adc\") "
Oct 30 22:11:10.376374 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:10.376150    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/233e0e6f-3447-4d9c-a443-0fa29a3b4adc-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/233e0e6f-3447-4d9c-a443-0fa29a3b4adc-workload-certs\") pod \"233e0e6f-3447-4d9c-a443-0fa29a3b4adc\" (UID: \"233e0e6f-3447-4d9c-a443-0fa29a3b4adc\") "
Oct 30 22:11:10.376374 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:10.376337    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/233e0e6f-3447-4d9c-a443-0fa29a3b4adc-istio-data" pod "233e0e6f-3447-4d9c-a443-0fa29a3b4adc" (UID: "233e0e6f-3447-4d9c-a443-0fa29a3b4adc"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:11:10.377328 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:10.376454    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/233e0e6f-3447-4d9c-a443-0fa29a3b4adc-workload-socket" pod "233e0e6f-3447-4d9c-a443-0fa29a3b4adc" (UID: "233e0e6f-3447-4d9c-a443-0fa29a3b4adc"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:11:10.377328 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:10.376812    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/233e0e6f-3447-4d9c-a443-0fa29a3b4adc-workload-certs" pod "233e0e6f-3447-4d9c-a443-0fa29a3b4adc" (UID: "233e0e6f-3447-4d9c-a443-0fa29a3b4adc"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:11:10.377328 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:10.377278    4473 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/233e0e6f-3447-4d9c-a443-0fa29a3b4adc-credential-socket\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:11:10.377328 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:10.377319    4473 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/233e0e6f-3447-4d9c-a443-0fa29a3b4adc-istio-data\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:11:10.377523 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:10.377337    4473 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/233e0e6f-3447-4d9c-a443-0fa29a3b4adc-workload-socket\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:11:10.377523 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:10.377354    4473 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/233e0e6f-3447-4d9c-a443-0fa29a3b4adc-workload-certs\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:11:10.377523 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:10.377367    4473 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/233e0e6f-3447-4d9c-a443-0fa29a3b4adc-istiod-ca-cert\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:11:10.378908 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:10.378882    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/233e0e6f-3447-4d9c-a443-0fa29a3b4adc-istio-envoy" pod "233e0e6f-3447-4d9c-a443-0fa29a3b4adc" (UID: "233e0e6f-3447-4d9c-a443-0fa29a3b4adc"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:11:10.379088 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:10.378973    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/233e0e6f-3447-4d9c-a443-0fa29a3b4adc-istio-token" pod "233e0e6f-3447-4d9c-a443-0fa29a3b4adc" (UID: "233e0e6f-3447-4d9c-a443-0fa29a3b4adc"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:11:10.379147 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:10.379119    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/233e0e6f-3447-4d9c-a443-0fa29a3b4adc-kube-api-access-t6sr7" pod "233e0e6f-3447-4d9c-a443-0fa29a3b4adc" (UID: "233e0e6f-3447-4d9c-a443-0fa29a3b4adc"). InnerVolumeSpecName "kube-api-access-t6sr7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:11:10.379187 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:10.379173    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/233e0e6f-3447-4d9c-a443-0fa29a3b4adc-istio-podinfo" pod "233e0e6f-3447-4d9c-a443-0fa29a3b4adc" (UID: "233e0e6f-3447-4d9c-a443-0fa29a3b4adc"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:11:10.436017 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:10.435971    4473 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-5d36f6-7dc89f4558-rwzdn"]
Oct 30 22:11:10.437267 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:10.437238    4473 cpu_manager.go:414] "containerMap: removing container" podUID="233e0e6f-3447-4d9c-a443-0fa29a3b4adc" containerName="istio-proxy"
Oct 30 22:11:10.437267 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:10.437266    4473 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="233e0e6f-3447-4d9c-a443-0fa29a3b4adc" containerName="istio-proxy"
Oct 30 22:11:10.437486 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:10.437319    4473 cpu_manager.go:414] "containerMap: removing container" podUID="233e0e6f-3447-4d9c-a443-0fa29a3b4adc" containerName="istio-proxy"
Oct 30 22:11:10.437486 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:10.437328    4473 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="233e0e6f-3447-4d9c-a443-0fa29a3b4adc" containerName="istio-proxy"
Oct 30 22:11:10.437555 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:10.437505    4473 memory_manager.go:365] "RemoveStaleState removing state" podUID="233e0e6f-3447-4d9c-a443-0fa29a3b4adc" containerName="istio-proxy"
Oct 30 22:11:10.437555 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:10.437526    4473 memory_manager.go:365] "RemoveStaleState removing state" podUID="233e0e6f-3447-4d9c-a443-0fa29a3b4adc" containerName="istio-proxy"
Oct 30 22:11:10.440817 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:10.440798    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-5d36f6-7dc89f4558-rwzdn"
Oct 30 22:11:10.468442 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:10.468405    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-5d36f6-7dc89f4558-rwzdn"]
Oct 30 22:11:10.479214 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:10.479185    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xf6kk\" (UniqueName: \"kubernetes.io/projected/fe2360b8-5fff-43b6-bfa7-9f3e86cdbbcf-kube-api-access-xf6kk\") pod \"maas-api-e2e-shared-a-w6-5d36f6-7dc89f4558-rwzdn\" (UID: \"fe2360b8-5fff-43b6-bfa7-9f3e86cdbbcf\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-5d36f6-7dc89f4558-rwzdn"
Oct 30 22:11:10.479401 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:10.479356    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fe2360b8-5fff-43b6-bfa7-9f3e86cdbbcf-maas-api-tls\") pod \"maas-api-e2e-shared-a-w6-5d36f6-7dc89f4558-rwzdn\" (UID: \"fe2360b8-5fff-43b6-bfa7-9f3e86cdbbcf\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-5d36f6-7dc89f4558-rwzdn"
Oct 30 22:11:10.479467 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:10.479426    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fe2360b8-5fff-43b6-bfa7-9f3e86cdbbcf-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w6-5d36f6-7dc89f4558-rwzdn\" (UID: \"fe2360b8-5fff-43b6-bfa7-9f3e86cdbbcf\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-5d36f6-7dc89f4558-rwzdn"
Oct 30 22:11:10.479516 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:10.479498    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fe2360b8-5fff-43b6-bfa7-9f3e86cdbbcf-metrics-certs\") pod \"maas-api-e2e-shared-a-w6-5d36f6-7dc89f4558-rwzdn\" (UID: \"fe2360b8-5fff-43b6-bfa7-9f3e86cdbbcf\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-5d36f6-7dc89f4558-rwzdn"
Oct 30 22:11:10.479698 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:10.479686    4473 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/233e0e6f-3447-4d9c-a443-0fa29a3b4adc-istio-envoy\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:11:10.479739 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:10.479703    4473 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t6sr7\" (UniqueName: \"kubernetes.io/projected/233e0e6f-3447-4d9c-a443-0fa29a3b4adc-kube-api-access-t6sr7\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:11:10.479739 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:10.479714    4473 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/233e0e6f-3447-4d9c-a443-0fa29a3b4adc-istio-token\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:11:10.479739 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:10.479723    4473 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/233e0e6f-3447-4d9c-a443-0fa29a3b4adc-istio-podinfo\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:11:10.496994 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:10.496963    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w6-5d36f6-serving-cert\""
Oct 30 22:11:10.580843 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:10.580746    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xf6kk\" (UniqueName: \"kubernetes.io/projected/fe2360b8-5fff-43b6-bfa7-9f3e86cdbbcf-kube-api-access-xf6kk\") pod \"maas-api-e2e-shared-a-w6-5d36f6-7dc89f4558-rwzdn\" (UID: \"fe2360b8-5fff-43b6-bfa7-9f3e86cdbbcf\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-5d36f6-7dc89f4558-rwzdn"
Oct 30 22:11:10.581033 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:10.580849    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fe2360b8-5fff-43b6-bfa7-9f3e86cdbbcf-maas-api-tls\") pod \"maas-api-e2e-shared-a-w6-5d36f6-7dc89f4558-rwzdn\" (UID: \"fe2360b8-5fff-43b6-bfa7-9f3e86cdbbcf\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-5d36f6-7dc89f4558-rwzdn"
Oct 30 22:11:10.581033 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:10.580879    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fe2360b8-5fff-43b6-bfa7-9f3e86cdbbcf-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w6-5d36f6-7dc89f4558-rwzdn\" (UID: \"fe2360b8-5fff-43b6-bfa7-9f3e86cdbbcf\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-5d36f6-7dc89f4558-rwzdn"
Oct 30 22:11:10.581033 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:11:10.580973    4473 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-5d36f6-serving-cert: secret "maas-api-e2e-shared-a-w6-5d36f6-serving-cert" not found
Oct 30 22:11:10.581033 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:10.581008    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fe2360b8-5fff-43b6-bfa7-9f3e86cdbbcf-metrics-certs\") pod \"maas-api-e2e-shared-a-w6-5d36f6-7dc89f4558-rwzdn\" (UID: \"fe2360b8-5fff-43b6-bfa7-9f3e86cdbbcf\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-5d36f6-7dc89f4558-rwzdn"
Oct 30 22:11:10.581241 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:11:10.581040    4473 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fe2360b8-5fff-43b6-bfa7-9f3e86cdbbcf-maas-api-tls podName:fe2360b8-5fff-43b6-bfa7-9f3e86cdbbcf nodeName:}" failed. No retries permitted until 2026-10-30 22:11:11.081018304 +0000 UTC m=+1716.854386794 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/fe2360b8-5fff-43b6-bfa7-9f3e86cdbbcf-maas-api-tls") pod "maas-api-e2e-shared-a-w6-5d36f6-7dc89f4558-rwzdn" (UID: "fe2360b8-5fff-43b6-bfa7-9f3e86cdbbcf") : secret "maas-api-e2e-shared-a-w6-5d36f6-serving-cert" not found
Oct 30 22:11:10.581618 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:10.581583    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fe2360b8-5fff-43b6-bfa7-9f3e86cdbbcf-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w6-5d36f6-7dc89f4558-rwzdn\" (UID: \"fe2360b8-5fff-43b6-bfa7-9f3e86cdbbcf\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-5d36f6-7dc89f4558-rwzdn"
Oct 30 22:11:10.583581 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:10.583559    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fe2360b8-5fff-43b6-bfa7-9f3e86cdbbcf-metrics-certs\") pod \"maas-api-e2e-shared-a-w6-5d36f6-7dc89f4558-rwzdn\" (UID: \"fe2360b8-5fff-43b6-bfa7-9f3e86cdbbcf\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-5d36f6-7dc89f4558-rwzdn"
Oct 30 22:11:10.639462 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:10.639428    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xf6kk\" (UniqueName: \"kubernetes.io/projected/fe2360b8-5fff-43b6-bfa7-9f3e86cdbbcf-kube-api-access-xf6kk\") pod \"maas-api-e2e-shared-a-w6-5d36f6-7dc89f4558-rwzdn\" (UID: \"fe2360b8-5fff-43b6-bfa7-9f3e86cdbbcf\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-5d36f6-7dc89f4558-rwzdn"
Oct 30 22:11:10.992702 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:10.992670    4473 generic.go:356] "Generic (PLEG): container finished" podID="233e0e6f-3447-4d9c-a443-0fa29a3b4adc" containerID="6216d1dac58cffb05954f1c0d99be4b05accde204017f6172781c223da299951" exitCode=0
Oct 30 22:11:10.992878 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:10.992743    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-497924bf-openshift-default-85499cb6f-fw82l"
Oct 30 22:11:10.992878 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:10.992754    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-497924bf-openshift-default-85499cb6f-fw82l" event={"ID":"233e0e6f-3447-4d9c-a443-0fa29a3b4adc","Type":"ContainerDied","Data":"6216d1dac58cffb05954f1c0d99be4b05accde204017f6172781c223da299951"}
Oct 30 22:11:10.992878 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:10.992841    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-497924bf-openshift-default-85499cb6f-fw82l" event={"ID":"233e0e6f-3447-4d9c-a443-0fa29a3b4adc","Type":"ContainerDied","Data":"668f8d3ebb09357385a8a3744bcb5cf84ce0e92a4a79676e74726b34e80395aa"}
Oct 30 22:11:10.992878 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:10.992866    4473 scope.go:122] "RemoveContainer" containerID="6216d1dac58cffb05954f1c0d99be4b05accde204017f6172781c223da299951"
Oct 30 22:11:11.004499 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:11.004483    4473 scope.go:122] "RemoveContainer" containerID="d155d219b8d5da74829b43cbb01d3e196699214e3ff6d95528e36803559bf89e"
Oct 30 22:11:11.015254 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:11.015230    4473 scope.go:122] "RemoveContainer" containerID="6216d1dac58cffb05954f1c0d99be4b05accde204017f6172781c223da299951"
Oct 30 22:11:11.015601 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:11:11.015575    4473 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6216d1dac58cffb05954f1c0d99be4b05accde204017f6172781c223da299951\": container with ID starting with 6216d1dac58cffb05954f1c0d99be4b05accde204017f6172781c223da299951 not found: ID does not exist" containerID="6216d1dac58cffb05954f1c0d99be4b05accde204017f6172781c223da299951"
Oct 30 22:11:11.015681 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:11.015611    4473 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6216d1dac58cffb05954f1c0d99be4b05accde204017f6172781c223da299951"} err="failed to get container status \"6216d1dac58cffb05954f1c0d99be4b05accde204017f6172781c223da299951\": rpc error: code = NotFound desc = could not find container \"6216d1dac58cffb05954f1c0d99be4b05accde204017f6172781c223da299951\": container with ID starting with 6216d1dac58cffb05954f1c0d99be4b05accde204017f6172781c223da299951 not found: ID does not exist"
Oct 30 22:11:11.015681 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:11.015632    4473 scope.go:122] "RemoveContainer" containerID="d155d219b8d5da74829b43cbb01d3e196699214e3ff6d95528e36803559bf89e"
Oct 30 22:11:11.015989 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:11:11.015961    4473 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d155d219b8d5da74829b43cbb01d3e196699214e3ff6d95528e36803559bf89e\": container with ID starting with d155d219b8d5da74829b43cbb01d3e196699214e3ff6d95528e36803559bf89e not found: ID does not exist" containerID="d155d219b8d5da74829b43cbb01d3e196699214e3ff6d95528e36803559bf89e"
Oct 30 22:11:11.016123 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:11.015994    4473 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d155d219b8d5da74829b43cbb01d3e196699214e3ff6d95528e36803559bf89e"} err="failed to get container status \"d155d219b8d5da74829b43cbb01d3e196699214e3ff6d95528e36803559bf89e\": rpc error: code = NotFound desc = could not find container \"d155d219b8d5da74829b43cbb01d3e196699214e3ff6d95528e36803559bf89e\": container with ID starting with d155d219b8d5da74829b43cbb01d3e196699214e3ff6d95528e36803559bf89e not found: ID does not exist"
Oct 30 22:11:11.125402 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:11.125358    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fe2360b8-5fff-43b6-bfa7-9f3e86cdbbcf-maas-api-tls\") pod \"maas-api-e2e-shared-a-w6-5d36f6-7dc89f4558-rwzdn\" (UID: \"fe2360b8-5fff-43b6-bfa7-9f3e86cdbbcf\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-5d36f6-7dc89f4558-rwzdn"
Oct 30 22:11:11.128157 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:11.128112    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fe2360b8-5fff-43b6-bfa7-9f3e86cdbbcf-maas-api-tls\") pod \"maas-api-e2e-shared-a-w6-5d36f6-7dc89f4558-rwzdn\" (UID: \"fe2360b8-5fff-43b6-bfa7-9f3e86cdbbcf\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-5d36f6-7dc89f4558-rwzdn"
Oct 30 22:11:11.165348 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:11.165316    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-497924bf-openshift-default-85499cb6f-fw82l"]
Oct 30 22:11:11.169543 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:11.169517    4473 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-497924bf-openshift-default-85499cb6f-fw82l"]
Oct 30 22:11:11.354675 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:11.354586    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-5d36f6-7dc89f4558-rwzdn"
Oct 30 22:11:11.751756 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:11.751710    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-5d36f6-7dc89f4558-rwzdn"]
Oct 30 22:11:11.751962 ip-10-0-140-18 kubenswrapper[4473]: W1030 22:11:11.751904    4473 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe2360b8_5fff_43b6_bfa7_9f3e86cdbbcf.slice/crio-3d26706bdf272189880de76b65ccd84a7a044040b188febcafef34d23afba73d WatchSource:0}: Error finding container 3d26706bdf272189880de76b65ccd84a7a044040b188febcafef34d23afba73d: Status 404 returned error can't find the container with id 3d26706bdf272189880de76b65ccd84a7a044040b188febcafef34d23afba73d
Oct 30 22:11:11.753725 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:11.753702    4473 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 22:11:12.002753 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:12.002660    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-5d36f6-7dc89f4558-rwzdn" event={"ID":"fe2360b8-5fff-43b6-bfa7-9f3e86cdbbcf","Type":"ContainerStarted","Data":"3d26706bdf272189880de76b65ccd84a7a044040b188febcafef34d23afba73d"}
Oct 30 22:11:12.361165 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:12.361081    4473 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="233e0e6f-3447-4d9c-a443-0fa29a3b4adc" path="/var/lib/kubelet/pods/233e0e6f-3447-4d9c-a443-0fa29a3b4adc/volumes"
Oct 30 22:11:13.053003 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:13.046441    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-5d36f6-7dc89f4558-rwzdn" event={"ID":"fe2360b8-5fff-43b6-bfa7-9f3e86cdbbcf","Type":"ContainerStarted","Data":"f57fc771af2403ebc282841fd6534b1bce7b2aeff06d77490cb0023f5b53db06"}
Oct 30 22:11:13.053003 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:13.048091    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-5d36f6-7dc89f4558-rwzdn"
Oct 30 22:11:13.181461 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:13.181395    4473 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-5d36f6-7dc89f4558-rwzdn" podStartSLOduration=2.389990758 podStartE2EDuration="3.181373474s" totalImagesPullingTime="791.382716ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:11:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:11:13.17727731 +0000 UTC m=+1718.950645801" watchObservedRunningTime="2026-10-30 22:11:13.181373474 +0000 UTC m=+1718.954741967"
Oct 30 22:11:13.382942 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:13.378061    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-5qn5b"]
Oct 30 22:11:13.382942 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:13.378341    4473 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-5qn5b" podUID="72b2d008-08fc-41aa-9515-ad4319bee895" containerName="istio-proxy" containerID="cri-o://782e2954c480d465a8cf704a93640bc929d32ce7e16c4fe7b38a514cc033d1ae" gracePeriod=30
Oct 30 22:11:14.669646 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:14.667640    4473 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-7c826c-9b5d559f7-dr9ng"]
Oct 30 22:11:14.680641 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:14.680610    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-7c826c-9b5d559f7-dr9ng"
Oct 30 22:11:14.713141 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:14.713089    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-7c826c-9b5d559f7-dr9ng"]
Oct 30 22:11:14.743026 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:14.741206    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w2-7c826c-dockercfg-fzt6t\""
Oct 30 22:11:14.802945 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:14.797650    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3e170928-4f15-43bf-9ad9-1a38b9bb0950-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-7c826c-9b5d559f7-dr9ng\" (UID: \"3e170928-4f15-43bf-9ad9-1a38b9bb0950\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-7c826c-9b5d559f7-dr9ng"
Oct 30 22:11:14.802945 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:14.798039    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dhvz8\" (UniqueName: \"kubernetes.io/projected/3e170928-4f15-43bf-9ad9-1a38b9bb0950-kube-api-access-dhvz8\") pod \"payload-processing-e2e-shared-b-w2-7c826c-9b5d559f7-dr9ng\" (UID: \"3e170928-4f15-43bf-9ad9-1a38b9bb0950\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-7c826c-9b5d559f7-dr9ng"
Oct 30 22:11:14.818215 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:14.806162    4473 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w2-7c826c\""
Oct 30 22:11:14.893012 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:14.892286    4473 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-7c826c-86f4595c8c-llmxr"]
Oct 30 22:11:14.907966 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:14.900588    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3e170928-4f15-43bf-9ad9-1a38b9bb0950-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-7c826c-9b5d559f7-dr9ng\" (UID: \"3e170928-4f15-43bf-9ad9-1a38b9bb0950\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-7c826c-9b5d559f7-dr9ng"
Oct 30 22:11:14.907966 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:14.900841    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dhvz8\" (UniqueName: \"kubernetes.io/projected/3e170928-4f15-43bf-9ad9-1a38b9bb0950-kube-api-access-dhvz8\") pod \"payload-processing-e2e-shared-b-w2-7c826c-9b5d559f7-dr9ng\" (UID: \"3e170928-4f15-43bf-9ad9-1a38b9bb0950\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-7c826c-9b5d559f7-dr9ng"
Oct 30 22:11:14.907966 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:14.902274    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3e170928-4f15-43bf-9ad9-1a38b9bb0950-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-7c826c-9b5d559f7-dr9ng\" (UID: \"3e170928-4f15-43bf-9ad9-1a38b9bb0950\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-7c826c-9b5d559f7-dr9ng"
Oct 30 22:11:14.909957 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:14.909850    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-7c826c-86f4595c8c-llmxr"
Oct 30 22:11:14.913087 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:14.910900    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-7c826c-86f4595c8c-llmxr"]
Oct 30 22:11:15.014038 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:15.004826    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-66crd\" (UniqueName: \"kubernetes.io/projected/1ff2c276-f027-4581-94b1-31ce87f60d59-kube-api-access-66crd\") pod \"payload-pre-processing-e2e-shared-b-w2-7c826c-86f4595c8c-llmxr\" (UID: \"1ff2c276-f027-4581-94b1-31ce87f60d59\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-7c826c-86f4595c8c-llmxr"
Oct 30 22:11:15.014038 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:15.004966    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1ff2c276-f027-4581-94b1-31ce87f60d59-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-7c826c-86f4595c8c-llmxr\" (UID: \"1ff2c276-f027-4581-94b1-31ce87f60d59\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-7c826c-86f4595c8c-llmxr"
Oct 30 22:11:15.014857 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:15.014831    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dhvz8\" (UniqueName: \"kubernetes.io/projected/3e170928-4f15-43bf-9ad9-1a38b9bb0950-kube-api-access-dhvz8\") pod \"payload-processing-e2e-shared-b-w2-7c826c-9b5d559f7-dr9ng\" (UID: \"3e170928-4f15-43bf-9ad9-1a38b9bb0950\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-7c826c-9b5d559f7-dr9ng"
Oct 30 22:11:15.023154 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:15.023100    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-7c826c-9b5d559f7-dr9ng"
Oct 30 22:11:15.128946 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:15.125055    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-66crd\" (UniqueName: \"kubernetes.io/projected/1ff2c276-f027-4581-94b1-31ce87f60d59-kube-api-access-66crd\") pod \"payload-pre-processing-e2e-shared-b-w2-7c826c-86f4595c8c-llmxr\" (UID: \"1ff2c276-f027-4581-94b1-31ce87f60d59\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-7c826c-86f4595c8c-llmxr"
Oct 30 22:11:15.164742 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:15.153092    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1ff2c276-f027-4581-94b1-31ce87f60d59-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-7c826c-86f4595c8c-llmxr\" (UID: \"1ff2c276-f027-4581-94b1-31ce87f60d59\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-7c826c-86f4595c8c-llmxr"
Oct 30 22:11:15.168523 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:15.162963    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1ff2c276-f027-4581-94b1-31ce87f60d59-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-7c826c-86f4595c8c-llmxr\" (UID: \"1ff2c276-f027-4581-94b1-31ce87f60d59\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-7c826c-86f4595c8c-llmxr"
Oct 30 22:11:15.206060 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:15.201599    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-66crd\" (UniqueName: \"kubernetes.io/projected/1ff2c276-f027-4581-94b1-31ce87f60d59-kube-api-access-66crd\") pod \"payload-pre-processing-e2e-shared-b-w2-7c826c-86f4595c8c-llmxr\" (UID: \"1ff2c276-f027-4581-94b1-31ce87f60d59\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-7c826c-86f4595c8c-llmxr"
Oct 30 22:11:15.226494 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:15.226461    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-7c826c-86f4595c8c-llmxr"
Oct 30 22:11:15.594805 ip-10-0-140-18 kubenswrapper[4473]: W1030 22:11:15.594759    4473 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e170928_4f15_43bf_9ad9_1a38b9bb0950.slice/crio-aeb86244822c1fb7166d69929e5bcd5b304f2b8eb15198c5073687e41494d082 WatchSource:0}: Error finding container aeb86244822c1fb7166d69929e5bcd5b304f2b8eb15198c5073687e41494d082: Status 404 returned error can't find the container with id aeb86244822c1fb7166d69929e5bcd5b304f2b8eb15198c5073687e41494d082
Oct 30 22:11:15.598087 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:15.598053    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-7c826c-9b5d559f7-dr9ng"]
Oct 30 22:11:15.646389 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:15.646355    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-7c826c-86f4595c8c-llmxr"]
Oct 30 22:11:15.647969 ip-10-0-140-18 kubenswrapper[4473]: W1030 22:11:15.647886    4473 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ff2c276_f027_4581_94b1_31ce87f60d59.slice/crio-9a7b5421a0bae2b56a8bead1c975c80b1d7aab19337ea85d8e5a85f4f710fe57 WatchSource:0}: Error finding container 9a7b5421a0bae2b56a8bead1c975c80b1d7aab19337ea85d8e5a85f4f710fe57: Status 404 returned error can't find the container with id 9a7b5421a0bae2b56a8bead1c975c80b1d7aab19337ea85d8e5a85f4f710fe57
Oct 30 22:11:16.210726 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:16.210683    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-7c826c-9b5d559f7-dr9ng" event={"ID":"3e170928-4f15-43bf-9ad9-1a38b9bb0950","Type":"ContainerStarted","Data":"4b9aceab0c6cef1020b7515554a5bde11aac311ea6253b7831704ed7e275d6ff"}
Oct 30 22:11:16.211231 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:16.210868    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-7c826c-9b5d559f7-dr9ng"
Oct 30 22:11:16.211231 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:16.210897    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-7c826c-9b5d559f7-dr9ng" event={"ID":"3e170928-4f15-43bf-9ad9-1a38b9bb0950","Type":"ContainerStarted","Data":"aeb86244822c1fb7166d69929e5bcd5b304f2b8eb15198c5073687e41494d082"}
Oct 30 22:11:16.212534 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:16.212502    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-7c826c-86f4595c8c-llmxr" event={"ID":"1ff2c276-f027-4581-94b1-31ce87f60d59","Type":"ContainerStarted","Data":"dc0af6b693a9399be3a1857b89f03a4f1090502ae70fc9a7bf64716c2017fec5"}
Oct 30 22:11:16.212654 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:16.212549    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-7c826c-86f4595c8c-llmxr" event={"ID":"1ff2c276-f027-4581-94b1-31ce87f60d59","Type":"ContainerStarted","Data":"9a7b5421a0bae2b56a8bead1c975c80b1d7aab19337ea85d8e5a85f4f710fe57"}
Oct 30 22:11:16.212654 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:16.212578    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-7c826c-86f4595c8c-llmxr"
Oct 30 22:11:16.330880 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:16.330821    4473 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-7c826c-9b5d559f7-dr9ng" podStartSLOduration=2.330801194 podStartE2EDuration="2.330801194s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:11:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:11:16.326471988 +0000 UTC m=+1722.099840475" watchObservedRunningTime="2026-10-30 22:11:16.330801194 +0000 UTC m=+1722.104169680"
Oct 30 22:11:16.472995 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:16.471182    4473 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-7c826c-86f4595c8c-llmxr" podStartSLOduration=2.471155543 podStartE2EDuration="2.471155543s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:11:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:11:16.455333624 +0000 UTC m=+1722.228702120" watchObservedRunningTime="2026-10-30 22:11:16.471155543 +0000 UTC m=+1722.244524031"
Oct 30 22:11:16.957841 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:16.957809    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w0-ebddcc-5f9dc96795-4z2w2"
Oct 30 22:11:16.958767 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:16.958743    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-ebddcc-75799c8f95-nqghx"
Oct 30 22:11:18.687383 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:18.687344    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-5qn5b"
Oct 30 22:11:18.843477 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:18.843375    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/72b2d008-08fc-41aa-9515-ad4319bee895-istio-token\" (UniqueName: \"kubernetes.io/projected/72b2d008-08fc-41aa-9515-ad4319bee895-istio-token\") pod \"72b2d008-08fc-41aa-9515-ad4319bee895\" (UID: \"72b2d008-08fc-41aa-9515-ad4319bee895\") "
Oct 30 22:11:18.843658 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:18.843538    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/72b2d008-08fc-41aa-9515-ad4319bee895-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/72b2d008-08fc-41aa-9515-ad4319bee895-istio-data\") pod \"72b2d008-08fc-41aa-9515-ad4319bee895\" (UID: \"72b2d008-08fc-41aa-9515-ad4319bee895\") "
Oct 30 22:11:18.843658 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:18.843577    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/72b2d008-08fc-41aa-9515-ad4319bee895-kube-api-access-j4rsj\" (UniqueName: \"kubernetes.io/projected/72b2d008-08fc-41aa-9515-ad4319bee895-kube-api-access-j4rsj\") pod \"72b2d008-08fc-41aa-9515-ad4319bee895\" (UID: \"72b2d008-08fc-41aa-9515-ad4319bee895\") "
Oct 30 22:11:18.843658 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:18.843618    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/72b2d008-08fc-41aa-9515-ad4319bee895-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/72b2d008-08fc-41aa-9515-ad4319bee895-workload-socket\") pod \"72b2d008-08fc-41aa-9515-ad4319bee895\" (UID: \"72b2d008-08fc-41aa-9515-ad4319bee895\") "
Oct 30 22:11:18.843819 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:18.843688    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/72b2d008-08fc-41aa-9515-ad4319bee895-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/72b2d008-08fc-41aa-9515-ad4319bee895-istio-podinfo\") pod \"72b2d008-08fc-41aa-9515-ad4319bee895\" (UID: \"72b2d008-08fc-41aa-9515-ad4319bee895\") "
Oct 30 22:11:18.843940 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:18.843889    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/72b2d008-08fc-41aa-9515-ad4319bee895-istio-data" pod "72b2d008-08fc-41aa-9515-ad4319bee895" (UID: "72b2d008-08fc-41aa-9515-ad4319bee895"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:11:18.844015 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:18.843909    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/72b2d008-08fc-41aa-9515-ad4319bee895-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/72b2d008-08fc-41aa-9515-ad4319bee895-workload-certs\") pod \"72b2d008-08fc-41aa-9515-ad4319bee895\" (UID: \"72b2d008-08fc-41aa-9515-ad4319bee895\") "
Oct 30 22:11:18.844067 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:18.844014    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/72b2d008-08fc-41aa-9515-ad4319bee895-workload-socket" pod "72b2d008-08fc-41aa-9515-ad4319bee895" (UID: "72b2d008-08fc-41aa-9515-ad4319bee895"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:11:18.844125 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:18.844101    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/72b2d008-08fc-41aa-9515-ad4319bee895-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/72b2d008-08fc-41aa-9515-ad4319bee895-istiod-ca-cert\") pod \"72b2d008-08fc-41aa-9515-ad4319bee895\" (UID: \"72b2d008-08fc-41aa-9515-ad4319bee895\") "
Oct 30 22:11:18.844304 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:18.844190    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/72b2d008-08fc-41aa-9515-ad4319bee895-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/72b2d008-08fc-41aa-9515-ad4319bee895-istio-envoy\") pod \"72b2d008-08fc-41aa-9515-ad4319bee895\" (UID: \"72b2d008-08fc-41aa-9515-ad4319bee895\") "
Oct 30 22:11:18.844304 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:18.844248    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/72b2d008-08fc-41aa-9515-ad4319bee895-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/72b2d008-08fc-41aa-9515-ad4319bee895-credential-socket\") pod \"72b2d008-08fc-41aa-9515-ad4319bee895\" (UID: \"72b2d008-08fc-41aa-9515-ad4319bee895\") "
Oct 30 22:11:18.844546 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:18.844502    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/72b2d008-08fc-41aa-9515-ad4319bee895-istiod-ca-cert" pod "72b2d008-08fc-41aa-9515-ad4319bee895" (UID: "72b2d008-08fc-41aa-9515-ad4319bee895"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:11:18.844790 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:18.844624    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/72b2d008-08fc-41aa-9515-ad4319bee895-workload-certs" pod "72b2d008-08fc-41aa-9515-ad4319bee895" (UID: "72b2d008-08fc-41aa-9515-ad4319bee895"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:11:18.844880 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:18.844792    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/72b2d008-08fc-41aa-9515-ad4319bee895-credential-socket" pod "72b2d008-08fc-41aa-9515-ad4319bee895" (UID: "72b2d008-08fc-41aa-9515-ad4319bee895"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:11:18.845198 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:18.845163    4473 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/72b2d008-08fc-41aa-9515-ad4319bee895-istiod-ca-cert\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:11:18.845198 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:18.845191    4473 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/72b2d008-08fc-41aa-9515-ad4319bee895-credential-socket\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:11:18.845300 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:18.845205    4473 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/72b2d008-08fc-41aa-9515-ad4319bee895-istio-data\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:11:18.845300 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:18.845219    4473 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/72b2d008-08fc-41aa-9515-ad4319bee895-workload-socket\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:11:18.845300 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:18.845232    4473 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/72b2d008-08fc-41aa-9515-ad4319bee895-workload-certs\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:11:18.846583 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:18.846560    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/72b2d008-08fc-41aa-9515-ad4319bee895-istio-token" pod "72b2d008-08fc-41aa-9515-ad4319bee895" (UID: "72b2d008-08fc-41aa-9515-ad4319bee895"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:11:18.846693 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:18.846596    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/72b2d008-08fc-41aa-9515-ad4319bee895-kube-api-access-j4rsj" pod "72b2d008-08fc-41aa-9515-ad4319bee895" (UID: "72b2d008-08fc-41aa-9515-ad4319bee895"). InnerVolumeSpecName "kube-api-access-j4rsj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:11:18.846891 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:18.846869    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/72b2d008-08fc-41aa-9515-ad4319bee895-istio-podinfo" pod "72b2d008-08fc-41aa-9515-ad4319bee895" (UID: "72b2d008-08fc-41aa-9515-ad4319bee895"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:11:18.846987 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:18.846968    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/72b2d008-08fc-41aa-9515-ad4319bee895-istio-envoy" pod "72b2d008-08fc-41aa-9515-ad4319bee895" (UID: "72b2d008-08fc-41aa-9515-ad4319bee895"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:11:18.946727 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:18.946688    4473 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/72b2d008-08fc-41aa-9515-ad4319bee895-istio-envoy\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:11:18.946727 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:18.946727    4473 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/72b2d008-08fc-41aa-9515-ad4319bee895-istio-token\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:11:18.947004 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:18.946742    4473 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j4rsj\" (UniqueName: \"kubernetes.io/projected/72b2d008-08fc-41aa-9515-ad4319bee895-kube-api-access-j4rsj\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:11:18.947004 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:18.946756    4473 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/72b2d008-08fc-41aa-9515-ad4319bee895-istio-podinfo\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:11:19.255077 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:19.255049    4473 generic.go:356] "Generic (PLEG): container finished" podID="72b2d008-08fc-41aa-9515-ad4319bee895" containerID="782e2954c480d465a8cf704a93640bc929d32ce7e16c4fe7b38a514cc033d1ae" exitCode=0
Oct 30 22:11:19.255266 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:19.255093    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-5qn5b"
Oct 30 22:11:19.255266 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:19.255129    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-5qn5b" event={"ID":"72b2d008-08fc-41aa-9515-ad4319bee895","Type":"ContainerDied","Data":"782e2954c480d465a8cf704a93640bc929d32ce7e16c4fe7b38a514cc033d1ae"}
Oct 30 22:11:19.255266 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:19.255190    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-5qn5b" event={"ID":"72b2d008-08fc-41aa-9515-ad4319bee895","Type":"ContainerDied","Data":"392fd41853a153eeb0c0047fe98bb58e2b3eadacec562ed9c6aaf542c6f9efca"}
Oct 30 22:11:19.255266 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:19.255212    4473 scope.go:122] "RemoveContainer" containerID="782e2954c480d465a8cf704a93640bc929d32ce7e16c4fe7b38a514cc033d1ae"
Oct 30 22:11:19.268065 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:19.268044    4473 scope.go:122] "RemoveContainer" containerID="782e2954c480d465a8cf704a93640bc929d32ce7e16c4fe7b38a514cc033d1ae"
Oct 30 22:11:19.268394 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:11:19.268373    4473 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"782e2954c480d465a8cf704a93640bc929d32ce7e16c4fe7b38a514cc033d1ae\": container with ID starting with 782e2954c480d465a8cf704a93640bc929d32ce7e16c4fe7b38a514cc033d1ae not found: ID does not exist" containerID="782e2954c480d465a8cf704a93640bc929d32ce7e16c4fe7b38a514cc033d1ae"
Oct 30 22:11:19.268505 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:19.268402    4473 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"782e2954c480d465a8cf704a93640bc929d32ce7e16c4fe7b38a514cc033d1ae"} err="failed to get container status \"782e2954c480d465a8cf704a93640bc929d32ce7e16c4fe7b38a514cc033d1ae\": rpc error: code = NotFound desc = could not find container \"782e2954c480d465a8cf704a93640bc929d32ce7e16c4fe7b38a514cc033d1ae\": container with ID starting with 782e2954c480d465a8cf704a93640bc929d32ce7e16c4fe7b38a514cc033d1ae not found: ID does not exist"
Oct 30 22:11:19.425295 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:19.425256    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-5qn5b"]
Oct 30 22:11:19.476195 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:19.476160    4473 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-5qn5b"]
Oct 30 22:11:20.206286 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:20.206258    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-5d36f6-7dc89f4558-rwzdn"
Oct 30 22:11:20.327350 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:11:20.327309    4473 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fncvg" containerName="extractor"
Oct 30 22:11:20.370459 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:20.370424    4473 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="72b2d008-08fc-41aa-9515-ad4319bee895" path="/var/lib/kubelet/pods/72b2d008-08fc-41aa-9515-ad4319bee895/volumes"
Oct 30 22:11:21.699944 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:21.695409    4473 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w5-e431fc-74d48cd559-c7bht"]
Oct 30 22:11:21.699944 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:21.697060    4473 cpu_manager.go:414] "containerMap: removing container" podUID="72b2d008-08fc-41aa-9515-ad4319bee895" containerName="istio-proxy"
Oct 30 22:11:21.699944 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:21.697085    4473 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="72b2d008-08fc-41aa-9515-ad4319bee895" containerName="istio-proxy"
Oct 30 22:11:21.699944 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:21.697389    4473 memory_manager.go:365] "RemoveStaleState removing state" podUID="72b2d008-08fc-41aa-9515-ad4319bee895" containerName="istio-proxy"
Oct 30 22:11:21.705949 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:21.702187    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w5-e431fc-74d48cd559-c7bht"
Oct 30 22:11:21.756643 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:21.756551    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w5-e431fc-74d48cd559-c7bht"]
Oct 30 22:11:21.870750 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:21.870717    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w5-e431fc-dockercfg-qfm46\""
Oct 30 22:11:21.871108 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:21.871084    4473 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w5-e431fc\""
Oct 30 22:11:22.046581 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:22.046491    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r56kc\" (UniqueName: \"kubernetes.io/projected/b78bb642-f809-4102-b4e5-ac485f1ca50d-kube-api-access-r56kc\") pod \"payload-pre-processing-e2e-shared-a-w5-e431fc-74d48cd559-c7bht\" (UID: \"b78bb642-f809-4102-b4e5-ac485f1ca50d\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w5-e431fc-74d48cd559-c7bht"
Oct 30 22:11:22.046759 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:22.046686    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b78bb642-f809-4102-b4e5-ac485f1ca50d-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w5-e431fc-74d48cd559-c7bht\" (UID: \"b78bb642-f809-4102-b4e5-ac485f1ca50d\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w5-e431fc-74d48cd559-c7bht"
Oct 30 22:11:22.184852 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:22.184818    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r56kc\" (UniqueName: \"kubernetes.io/projected/b78bb642-f809-4102-b4e5-ac485f1ca50d-kube-api-access-r56kc\") pod \"payload-pre-processing-e2e-shared-a-w5-e431fc-74d48cd559-c7bht\" (UID: \"b78bb642-f809-4102-b4e5-ac485f1ca50d\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w5-e431fc-74d48cd559-c7bht"
Oct 30 22:11:22.208565 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:22.196686    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b78bb642-f809-4102-b4e5-ac485f1ca50d-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w5-e431fc-74d48cd559-c7bht\" (UID: \"b78bb642-f809-4102-b4e5-ac485f1ca50d\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w5-e431fc-74d48cd559-c7bht"
Oct 30 22:11:22.208565 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:22.204999    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b78bb642-f809-4102-b4e5-ac485f1ca50d-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w5-e431fc-74d48cd559-c7bht\" (UID: \"b78bb642-f809-4102-b4e5-ac485f1ca50d\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w5-e431fc-74d48cd559-c7bht"
Oct 30 22:11:22.305180 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:22.305104    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r56kc\" (UniqueName: \"kubernetes.io/projected/b78bb642-f809-4102-b4e5-ac485f1ca50d-kube-api-access-r56kc\") pod \"payload-pre-processing-e2e-shared-a-w5-e431fc-74d48cd559-c7bht\" (UID: \"b78bb642-f809-4102-b4e5-ac485f1ca50d\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w5-e431fc-74d48cd559-c7bht"
Oct 30 22:11:22.327947 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:22.327884    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w5-e431fc-74d48cd559-c7bht"
Oct 30 22:11:22.714138 ip-10-0-140-18 kubenswrapper[4473]: W1030 22:11:22.714111    4473 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb78bb642_f809_4102_b4e5_ac485f1ca50d.slice/crio-a3baf5317476b04f24191ae9448be093ff92bf87b9ede8ca8ea767dadd525aa3 WatchSource:0}: Error finding container a3baf5317476b04f24191ae9448be093ff92bf87b9ede8ca8ea767dadd525aa3: Status 404 returned error can't find the container with id a3baf5317476b04f24191ae9448be093ff92bf87b9ede8ca8ea767dadd525aa3
Oct 30 22:11:22.714540 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:22.714444    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w5-e431fc-74d48cd559-c7bht"]
Oct 30 22:11:23.317579 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:23.317537    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w5-e431fc-74d48cd559-c7bht" event={"ID":"b78bb642-f809-4102-b4e5-ac485f1ca50d","Type":"ContainerStarted","Data":"ee015d5000a27c4f2af5043d5c85626434a172cf344e7389e85bfeb54dd244c0"}
Oct 30 22:11:23.317766 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:23.317605    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w5-e431fc-74d48cd559-c7bht" event={"ID":"b78bb642-f809-4102-b4e5-ac485f1ca50d","Type":"ContainerStarted","Data":"a3baf5317476b04f24191ae9448be093ff92bf87b9ede8ca8ea767dadd525aa3"}
Oct 30 22:11:23.317766 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:23.317634    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w5-e431fc-74d48cd559-c7bht"
Oct 30 22:11:23.441002 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:23.440947    4473 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w5-e431fc-74d48cd559-c7bht" podStartSLOduration=2.4409300910000002 podStartE2EDuration="2.440930091s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:11:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:11:23.439220274 +0000 UTC m=+1729.212588761" watchObservedRunningTime="2026-10-30 22:11:23.440930091 +0000 UTC m=+1729.214298606"
Oct 30 22:11:27.262982 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:27.261221    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-7c826c-9b5d559f7-dr9ng"
Oct 30 22:11:27.275995 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:27.265823    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-7c826c-86f4595c8c-llmxr"
Oct 30 22:11:34.328633 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:34.328599    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w5-e431fc-74d48cd559-c7bht"
Oct 30 22:11:42.981944 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:42.981894    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-gsc42"]
Oct 30 22:11:42.982386 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:42.982161    4473 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-gsc42" podUID="3c982f07-a1d8-4b18-a065-4e53f7e5022a" containerName="istio-proxy" containerID="cri-o://ce6d303a42496486091e3e9f8d8f2e2895ecdcbfe30841234474479823fceed3" gracePeriod=30
Oct 30 22:11:43.032245 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:43.032211    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-jsdrm"]
Oct 30 22:11:43.032452 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:43.032432    4473 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-jsdrm" podUID="f100a752-adf7-4593-b9cb-9a4b276473c4" containerName="istio-proxy" containerID="cri-o://de4d576a5ae5e7ccba6f1ff4d4c50cb825c54004dc84421151be2927ad215739" gracePeriod=30
Oct 30 22:11:43.036169 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:43.036142    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-qng2w"]
Oct 30 22:11:43.036368 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:43.036340    4473 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-qng2w" podUID="6560c1bc-b507-4a92-8586-23e0fcc5362a" containerName="istio-proxy" containerID="cri-o://d26e70139b180333d21d0d8011cf38c246c5461712eb9266e0a20acae0653eae" gracePeriod=30
Oct 30 22:11:43.083704 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:43.083668    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-ktzwz"]
Oct 30 22:11:43.084176 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:43.084146    4473 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-ktzwz" podUID="df9bbdb1-5a45-4223-b03a-5d5e11542e60" containerName="istio-proxy" containerID="cri-o://db07e6e36250d9ebf8ae105051d349e432a4d0ba0d0eaecfe66b14d4a53c972c" gracePeriod=30
Oct 30 22:11:46.720751 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:46.720666    4473 patch_prober.go:29] interesting pod/e2e-worker-w2-0c99b1-openshift-default-967855977-jsdrm container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 22:11:46.720751 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:46.720724    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-jsdrm" podUID="f100a752-adf7-4593-b9cb-9a4b276473c4" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 22:11:48.842136 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:48.840826    4473 generic.go:356] "Generic (PLEG): container finished" podID="6560c1bc-b507-4a92-8586-23e0fcc5362a" containerID="d26e70139b180333d21d0d8011cf38c246c5461712eb9266e0a20acae0653eae" exitCode=0
Oct 30 22:11:48.842136 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:48.840993    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-qng2w" event={"ID":"6560c1bc-b507-4a92-8586-23e0fcc5362a","Type":"ContainerDied","Data":"d26e70139b180333d21d0d8011cf38c246c5461712eb9266e0a20acae0653eae"}
Oct 30 22:11:48.842136 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:48.841053    4473 scope.go:122] "RemoveContainer" containerID="66d5162ae5be3f6186045d73c5997ea750a278198e32e17fc0b5624548b96834"
Oct 30 22:11:48.866062 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:48.863109    4473 generic.go:356] "Generic (PLEG): container finished" podID="f100a752-adf7-4593-b9cb-9a4b276473c4" containerID="de4d576a5ae5e7ccba6f1ff4d4c50cb825c54004dc84421151be2927ad215739" exitCode=0
Oct 30 22:11:48.866062 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:48.863263    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-jsdrm" event={"ID":"f100a752-adf7-4593-b9cb-9a4b276473c4","Type":"ContainerDied","Data":"de4d576a5ae5e7ccba6f1ff4d4c50cb825c54004dc84421151be2927ad215739"}
Oct 30 22:11:48.894505 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:48.890736    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-ktzwz" event={"ID":"df9bbdb1-5a45-4223-b03a-5d5e11542e60","Type":"ContainerDied","Data":"db07e6e36250d9ebf8ae105051d349e432a4d0ba0d0eaecfe66b14d4a53c972c"}
Oct 30 22:11:48.894505 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:48.890309    4473 generic.go:356] "Generic (PLEG): container finished" podID="df9bbdb1-5a45-4223-b03a-5d5e11542e60" containerID="db07e6e36250d9ebf8ae105051d349e432a4d0ba0d0eaecfe66b14d4a53c972c" exitCode=0
Oct 30 22:11:48.894505 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:48.894145    4473 generic.go:356] "Generic (PLEG): container finished" podID="3c982f07-a1d8-4b18-a065-4e53f7e5022a" containerID="ce6d303a42496486091e3e9f8d8f2e2895ecdcbfe30841234474479823fceed3" exitCode=0
Oct 30 22:11:48.894505 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:48.894218    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-gsc42" event={"ID":"3c982f07-a1d8-4b18-a065-4e53f7e5022a","Type":"ContainerDied","Data":"ce6d303a42496486091e3e9f8d8f2e2895ecdcbfe30841234474479823fceed3"}
Oct 30 22:11:48.901491 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:48.901259    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-qng2w"
Oct 30 22:11:48.922191 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:48.920468    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6560c1bc-b507-4a92-8586-23e0fcc5362a-kube-api-access-qpsj6\" (UniqueName: \"kubernetes.io/projected/6560c1bc-b507-4a92-8586-23e0fcc5362a-kube-api-access-qpsj6\") pod \"6560c1bc-b507-4a92-8586-23e0fcc5362a\" (UID: \"6560c1bc-b507-4a92-8586-23e0fcc5362a\") "
Oct 30 22:11:48.922191 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:48.920534    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6560c1bc-b507-4a92-8586-23e0fcc5362a-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6560c1bc-b507-4a92-8586-23e0fcc5362a-workload-certs\") pod \"6560c1bc-b507-4a92-8586-23e0fcc5362a\" (UID: \"6560c1bc-b507-4a92-8586-23e0fcc5362a\") "
Oct 30 22:11:48.922191 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:48.920567    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6560c1bc-b507-4a92-8586-23e0fcc5362a-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6560c1bc-b507-4a92-8586-23e0fcc5362a-istio-data\") pod \"6560c1bc-b507-4a92-8586-23e0fcc5362a\" (UID: \"6560c1bc-b507-4a92-8586-23e0fcc5362a\") "
Oct 30 22:11:48.922191 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:48.920708    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6560c1bc-b507-4a92-8586-23e0fcc5362a-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6560c1bc-b507-4a92-8586-23e0fcc5362a-istiod-ca-cert\") pod \"6560c1bc-b507-4a92-8586-23e0fcc5362a\" (UID: \"6560c1bc-b507-4a92-8586-23e0fcc5362a\") "
Oct 30 22:11:48.922191 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:48.920755    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6560c1bc-b507-4a92-8586-23e0fcc5362a-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6560c1bc-b507-4a92-8586-23e0fcc5362a-istio-envoy\") pod \"6560c1bc-b507-4a92-8586-23e0fcc5362a\" (UID: \"6560c1bc-b507-4a92-8586-23e0fcc5362a\") "
Oct 30 22:11:48.922191 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:48.920863    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/6560c1bc-b507-4a92-8586-23e0fcc5362a-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6560c1bc-b507-4a92-8586-23e0fcc5362a-istio-podinfo\") pod \"6560c1bc-b507-4a92-8586-23e0fcc5362a\" (UID: \"6560c1bc-b507-4a92-8586-23e0fcc5362a\") "
Oct 30 22:11:48.922191 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:48.920893    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6560c1bc-b507-4a92-8586-23e0fcc5362a-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6560c1bc-b507-4a92-8586-23e0fcc5362a-workload-socket\") pod \"6560c1bc-b507-4a92-8586-23e0fcc5362a\" (UID: \"6560c1bc-b507-4a92-8586-23e0fcc5362a\") "
Oct 30 22:11:48.922191 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:48.920981    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6560c1bc-b507-4a92-8586-23e0fcc5362a-istio-token\" (UniqueName: \"kubernetes.io/projected/6560c1bc-b507-4a92-8586-23e0fcc5362a-istio-token\") pod \"6560c1bc-b507-4a92-8586-23e0fcc5362a\" (UID: \"6560c1bc-b507-4a92-8586-23e0fcc5362a\") "
Oct 30 22:11:48.922191 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:48.921053    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6560c1bc-b507-4a92-8586-23e0fcc5362a-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6560c1bc-b507-4a92-8586-23e0fcc5362a-credential-socket\") pod \"6560c1bc-b507-4a92-8586-23e0fcc5362a\" (UID: \"6560c1bc-b507-4a92-8586-23e0fcc5362a\") "
Oct 30 22:11:48.922785 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:48.922371    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6560c1bc-b507-4a92-8586-23e0fcc5362a-credential-socket" pod "6560c1bc-b507-4a92-8586-23e0fcc5362a" (UID: "6560c1bc-b507-4a92-8586-23e0fcc5362a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:11:48.923384 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:48.923091    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6560c1bc-b507-4a92-8586-23e0fcc5362a-workload-certs" pod "6560c1bc-b507-4a92-8586-23e0fcc5362a" (UID: "6560c1bc-b507-4a92-8586-23e0fcc5362a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:11:48.931141 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:48.928814    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6560c1bc-b507-4a92-8586-23e0fcc5362a-istio-data" pod "6560c1bc-b507-4a92-8586-23e0fcc5362a" (UID: "6560c1bc-b507-4a92-8586-23e0fcc5362a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:11:48.931141 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:48.929054    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6560c1bc-b507-4a92-8586-23e0fcc5362a-workload-socket" pod "6560c1bc-b507-4a92-8586-23e0fcc5362a" (UID: "6560c1bc-b507-4a92-8586-23e0fcc5362a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:11:48.931141 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:48.929088    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6560c1bc-b507-4a92-8586-23e0fcc5362a-istiod-ca-cert" pod "6560c1bc-b507-4a92-8586-23e0fcc5362a" (UID: "6560c1bc-b507-4a92-8586-23e0fcc5362a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:11:48.934963 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:48.932419    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6560c1bc-b507-4a92-8586-23e0fcc5362a-istio-podinfo" pod "6560c1bc-b507-4a92-8586-23e0fcc5362a" (UID: "6560c1bc-b507-4a92-8586-23e0fcc5362a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:11:48.963945 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:48.957547    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6560c1bc-b507-4a92-8586-23e0fcc5362a-istio-envoy" pod "6560c1bc-b507-4a92-8586-23e0fcc5362a" (UID: "6560c1bc-b507-4a92-8586-23e0fcc5362a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:11:48.963945 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:48.958113    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6560c1bc-b507-4a92-8586-23e0fcc5362a-istio-token" pod "6560c1bc-b507-4a92-8586-23e0fcc5362a" (UID: "6560c1bc-b507-4a92-8586-23e0fcc5362a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:11:48.963945 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:48.958717    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6560c1bc-b507-4a92-8586-23e0fcc5362a-kube-api-access-qpsj6" pod "6560c1bc-b507-4a92-8586-23e0fcc5362a" (UID: "6560c1bc-b507-4a92-8586-23e0fcc5362a"). InnerVolumeSpecName "kube-api-access-qpsj6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:11:49.024290 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:49.024251    4473 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6560c1bc-b507-4a92-8586-23e0fcc5362a-istiod-ca-cert\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:11:49.024290 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:49.024293    4473 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6560c1bc-b507-4a92-8586-23e0fcc5362a-istio-envoy\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:11:49.024486 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:49.024309    4473 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6560c1bc-b507-4a92-8586-23e0fcc5362a-istio-podinfo\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:11:49.024486 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:49.024322    4473 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6560c1bc-b507-4a92-8586-23e0fcc5362a-workload-socket\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:11:49.024486 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:49.024336    4473 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6560c1bc-b507-4a92-8586-23e0fcc5362a-istio-token\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:11:49.024486 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:49.024349    4473 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6560c1bc-b507-4a92-8586-23e0fcc5362a-credential-socket\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:11:49.024486 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:49.024363    4473 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qpsj6\" (UniqueName: \"kubernetes.io/projected/6560c1bc-b507-4a92-8586-23e0fcc5362a-kube-api-access-qpsj6\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:11:49.024486 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:49.024378    4473 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6560c1bc-b507-4a92-8586-23e0fcc5362a-workload-certs\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:11:49.024486 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:49.024391    4473 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6560c1bc-b507-4a92-8586-23e0fcc5362a-istio-data\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:11:49.111535 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:49.111144    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-gsc42"
Oct 30 22:11:49.135956 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:49.125070    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3c982f07-a1d8-4b18-a065-4e53f7e5022a-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3c982f07-a1d8-4b18-a065-4e53f7e5022a-istio-envoy\") pod \"3c982f07-a1d8-4b18-a065-4e53f7e5022a\" (UID: \"3c982f07-a1d8-4b18-a065-4e53f7e5022a\") "
Oct 30 22:11:49.165949 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:49.164356    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c982f07-a1d8-4b18-a065-4e53f7e5022a-istio-envoy" pod "3c982f07-a1d8-4b18-a065-4e53f7e5022a" (UID: "3c982f07-a1d8-4b18-a065-4e53f7e5022a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:11:49.166165 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:49.165976    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3c982f07-a1d8-4b18-a065-4e53f7e5022a-kube-api-access-2jgnc\" (UniqueName: \"kubernetes.io/projected/3c982f07-a1d8-4b18-a065-4e53f7e5022a-kube-api-access-2jgnc\") pod \"3c982f07-a1d8-4b18-a065-4e53f7e5022a\" (UID: \"3c982f07-a1d8-4b18-a065-4e53f7e5022a\") "
Oct 30 22:11:49.166165 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:49.166144    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3c982f07-a1d8-4b18-a065-4e53f7e5022a-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c982f07-a1d8-4b18-a065-4e53f7e5022a-workload-socket\") pod \"3c982f07-a1d8-4b18-a065-4e53f7e5022a\" (UID: \"3c982f07-a1d8-4b18-a065-4e53f7e5022a\") "
Oct 30 22:11:49.166356 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:49.166244    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3c982f07-a1d8-4b18-a065-4e53f7e5022a-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3c982f07-a1d8-4b18-a065-4e53f7e5022a-istiod-ca-cert\") pod \"3c982f07-a1d8-4b18-a065-4e53f7e5022a\" (UID: \"3c982f07-a1d8-4b18-a065-4e53f7e5022a\") "
Oct 30 22:11:49.166356 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:49.166327    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3c982f07-a1d8-4b18-a065-4e53f7e5022a-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c982f07-a1d8-4b18-a065-4e53f7e5022a-credential-socket\") pod \"3c982f07-a1d8-4b18-a065-4e53f7e5022a\" (UID: \"3c982f07-a1d8-4b18-a065-4e53f7e5022a\") "
Oct 30 22:11:49.166514 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:49.166405    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3c982f07-a1d8-4b18-a065-4e53f7e5022a-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3c982f07-a1d8-4b18-a065-4e53f7e5022a-istio-data\") pod \"3c982f07-a1d8-4b18-a065-4e53f7e5022a\" (UID: \"3c982f07-a1d8-4b18-a065-4e53f7e5022a\") "
Oct 30 22:11:49.166514 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:49.166448    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3c982f07-a1d8-4b18-a065-4e53f7e5022a-istio-token\" (UniqueName: \"kubernetes.io/projected/3c982f07-a1d8-4b18-a065-4e53f7e5022a-istio-token\") pod \"3c982f07-a1d8-4b18-a065-4e53f7e5022a\" (UID: \"3c982f07-a1d8-4b18-a065-4e53f7e5022a\") "
Oct 30 22:11:49.166679 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:49.166510    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3c982f07-a1d8-4b18-a065-4e53f7e5022a-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3c982f07-a1d8-4b18-a065-4e53f7e5022a-workload-certs\") pod \"3c982f07-a1d8-4b18-a065-4e53f7e5022a\" (UID: \"3c982f07-a1d8-4b18-a065-4e53f7e5022a\") "
Oct 30 22:11:49.166679 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:49.166648    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/3c982f07-a1d8-4b18-a065-4e53f7e5022a-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3c982f07-a1d8-4b18-a065-4e53f7e5022a-istio-podinfo\") pod \"3c982f07-a1d8-4b18-a065-4e53f7e5022a\" (UID: \"3c982f07-a1d8-4b18-a065-4e53f7e5022a\") "
Oct 30 22:11:49.166679 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:49.166666    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c982f07-a1d8-4b18-a065-4e53f7e5022a-workload-socket" pod "3c982f07-a1d8-4b18-a065-4e53f7e5022a" (UID: "3c982f07-a1d8-4b18-a065-4e53f7e5022a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:11:49.167322 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:49.166680    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c982f07-a1d8-4b18-a065-4e53f7e5022a-credential-socket" pod "3c982f07-a1d8-4b18-a065-4e53f7e5022a" (UID: "3c982f07-a1d8-4b18-a065-4e53f7e5022a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:11:49.167322 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:49.166993    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3c982f07-a1d8-4b18-a065-4e53f7e5022a-istiod-ca-cert" pod "3c982f07-a1d8-4b18-a065-4e53f7e5022a" (UID: "3c982f07-a1d8-4b18-a065-4e53f7e5022a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:11:49.167322 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:49.167247    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c982f07-a1d8-4b18-a065-4e53f7e5022a-istio-data" pod "3c982f07-a1d8-4b18-a065-4e53f7e5022a" (UID: "3c982f07-a1d8-4b18-a065-4e53f7e5022a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:11:49.167831 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:49.167651    4473 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3c982f07-a1d8-4b18-a065-4e53f7e5022a-istiod-ca-cert\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:11:49.167831 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:49.167675    4473 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c982f07-a1d8-4b18-a065-4e53f7e5022a-credential-socket\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:11:49.167831 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:49.167690    4473 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3c982f07-a1d8-4b18-a065-4e53f7e5022a-istio-data\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:11:49.167831 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:49.167718    4473 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3c982f07-a1d8-4b18-a065-4e53f7e5022a-istio-envoy\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:11:49.167831 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:49.167734    4473 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c982f07-a1d8-4b18-a065-4e53f7e5022a-workload-socket\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:11:49.169024 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:49.168866    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3c982f07-a1d8-4b18-a065-4e53f7e5022a-kube-api-access-2jgnc" pod "3c982f07-a1d8-4b18-a065-4e53f7e5022a" (UID: "3c982f07-a1d8-4b18-a065-4e53f7e5022a"). InnerVolumeSpecName "kube-api-access-2jgnc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:11:49.169591 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:49.169564    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c982f07-a1d8-4b18-a065-4e53f7e5022a-workload-certs" pod "3c982f07-a1d8-4b18-a065-4e53f7e5022a" (UID: "3c982f07-a1d8-4b18-a065-4e53f7e5022a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:11:49.171943 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:49.170480    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3c982f07-a1d8-4b18-a065-4e53f7e5022a-istio-podinfo" pod "3c982f07-a1d8-4b18-a065-4e53f7e5022a" (UID: "3c982f07-a1d8-4b18-a065-4e53f7e5022a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:11:49.171943 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:49.171168    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3c982f07-a1d8-4b18-a065-4e53f7e5022a-istio-token" pod "3c982f07-a1d8-4b18-a065-4e53f7e5022a" (UID: "3c982f07-a1d8-4b18-a065-4e53f7e5022a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:11:49.179843 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:49.179818    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-ktzwz"
Oct 30 22:11:49.184010 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:49.183987    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-jsdrm"
Oct 30 22:11:49.269030 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:49.268889    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f100a752-adf7-4593-b9cb-9a4b276473c4-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f100a752-adf7-4593-b9cb-9a4b276473c4-istio-envoy\") pod \"f100a752-adf7-4593-b9cb-9a4b276473c4\" (UID: \"f100a752-adf7-4593-b9cb-9a4b276473c4\") "
Oct 30 22:11:49.269218 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:49.269176    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/df9bbdb1-5a45-4223-b03a-5d5e11542e60-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/df9bbdb1-5a45-4223-b03a-5d5e11542e60-istiod-ca-cert\") pod \"df9bbdb1-5a45-4223-b03a-5d5e11542e60\" (UID: \"df9bbdb1-5a45-4223-b03a-5d5e11542e60\") "
Oct 30 22:11:49.269218 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:49.269209    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f100a752-adf7-4593-b9cb-9a4b276473c4-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f100a752-adf7-4593-b9cb-9a4b276473c4-workload-certs\") pod \"f100a752-adf7-4593-b9cb-9a4b276473c4\" (UID: \"f100a752-adf7-4593-b9cb-9a4b276473c4\") "
Oct 30 22:11:49.269365 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:49.269241    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/df9bbdb1-5a45-4223-b03a-5d5e11542e60-kube-api-access-t9px6\" (UniqueName: \"kubernetes.io/projected/df9bbdb1-5a45-4223-b03a-5d5e11542e60-kube-api-access-t9px6\") pod \"df9bbdb1-5a45-4223-b03a-5d5e11542e60\" (UID: \"df9bbdb1-5a45-4223-b03a-5d5e11542e60\") "
Oct 30 22:11:49.269365 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:49.269279    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/df9bbdb1-5a45-4223-b03a-5d5e11542e60-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/df9bbdb1-5a45-4223-b03a-5d5e11542e60-workload-socket\") pod \"df9bbdb1-5a45-4223-b03a-5d5e11542e60\" (UID: \"df9bbdb1-5a45-4223-b03a-5d5e11542e60\") "
Oct 30 22:11:49.269365 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:49.269312    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/df9bbdb1-5a45-4223-b03a-5d5e11542e60-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/df9bbdb1-5a45-4223-b03a-5d5e11542e60-istio-podinfo\") pod \"df9bbdb1-5a45-4223-b03a-5d5e11542e60\" (UID: \"df9bbdb1-5a45-4223-b03a-5d5e11542e60\") "
Oct 30 22:11:49.269365 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:49.269333    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f100a752-adf7-4593-b9cb-9a4b276473c4-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f100a752-adf7-4593-b9cb-9a4b276473c4-credential-socket\") pod \"f100a752-adf7-4593-b9cb-9a4b276473c4\" (UID: \"f100a752-adf7-4593-b9cb-9a4b276473c4\") "
Oct 30 22:11:49.269365 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:49.269366    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/df9bbdb1-5a45-4223-b03a-5d5e11542e60-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/df9bbdb1-5a45-4223-b03a-5d5e11542e60-istio-envoy\") pod \"df9bbdb1-5a45-4223-b03a-5d5e11542e60\" (UID: \"df9bbdb1-5a45-4223-b03a-5d5e11542e60\") "
Oct 30 22:11:49.269554 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:49.269397    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f100a752-adf7-4593-b9cb-9a4b276473c4-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f100a752-adf7-4593-b9cb-9a4b276473c4-istio-data\") pod \"f100a752-adf7-4593-b9cb-9a4b276473c4\" (UID: \"f100a752-adf7-4593-b9cb-9a4b276473c4\") "
Oct 30 22:11:49.269554 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:49.269422    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f100a752-adf7-4593-b9cb-9a4b276473c4-istio-token\" (UniqueName: \"kubernetes.io/projected/f100a752-adf7-4593-b9cb-9a4b276473c4-istio-token\") pod \"f100a752-adf7-4593-b9cb-9a4b276473c4\" (UID: \"f100a752-adf7-4593-b9cb-9a4b276473c4\") "
Oct 30 22:11:49.269554 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:49.269445    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f100a752-adf7-4593-b9cb-9a4b276473c4-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f100a752-adf7-4593-b9cb-9a4b276473c4-istiod-ca-cert\") pod \"f100a752-adf7-4593-b9cb-9a4b276473c4\" (UID: \"f100a752-adf7-4593-b9cb-9a4b276473c4\") "
Oct 30 22:11:49.269554 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:49.269460    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/df9bbdb1-5a45-4223-b03a-5d5e11542e60-istio-token\" (UniqueName: \"kubernetes.io/projected/df9bbdb1-5a45-4223-b03a-5d5e11542e60-istio-token\") pod \"df9bbdb1-5a45-4223-b03a-5d5e11542e60\" (UID: \"df9bbdb1-5a45-4223-b03a-5d5e11542e60\") "
Oct 30 22:11:49.269554 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:49.269476    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/df9bbdb1-5a45-4223-b03a-5d5e11542e60-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/df9bbdb1-5a45-4223-b03a-5d5e11542e60-credential-socket\") pod \"df9bbdb1-5a45-4223-b03a-5d5e11542e60\" (UID: \"df9bbdb1-5a45-4223-b03a-5d5e11542e60\") "
Oct 30 22:11:49.269554 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:49.269476    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f100a752-adf7-4593-b9cb-9a4b276473c4-workload-certs" pod "f100a752-adf7-4593-b9cb-9a4b276473c4" (UID: "f100a752-adf7-4593-b9cb-9a4b276473c4"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:11:49.269554 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:49.269504    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f100a752-adf7-4593-b9cb-9a4b276473c4-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f100a752-adf7-4593-b9cb-9a4b276473c4-workload-socket\") pod \"f100a752-adf7-4593-b9cb-9a4b276473c4\" (UID: \"f100a752-adf7-4593-b9cb-9a4b276473c4\") "
Oct 30 22:11:49.269789 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:49.269571    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/df9bbdb1-5a45-4223-b03a-5d5e11542e60-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/df9bbdb1-5a45-4223-b03a-5d5e11542e60-istio-data\") pod \"df9bbdb1-5a45-4223-b03a-5d5e11542e60\" (UID: \"df9bbdb1-5a45-4223-b03a-5d5e11542e60\") "
Oct 30 22:11:49.269789 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:49.269595    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/df9bbdb1-5a45-4223-b03a-5d5e11542e60-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/df9bbdb1-5a45-4223-b03a-5d5e11542e60-workload-certs\") pod \"df9bbdb1-5a45-4223-b03a-5d5e11542e60\" (UID: \"df9bbdb1-5a45-4223-b03a-5d5e11542e60\") "
Oct 30 22:11:49.269789 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:49.269601    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/df9bbdb1-5a45-4223-b03a-5d5e11542e60-istiod-ca-cert" pod "df9bbdb1-5a45-4223-b03a-5d5e11542e60" (UID: "df9bbdb1-5a45-4223-b03a-5d5e11542e60"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:11:49.269789 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:49.269614    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f100a752-adf7-4593-b9cb-9a4b276473c4-kube-api-access-zt5mt\" (UniqueName: \"kubernetes.io/projected/f100a752-adf7-4593-b9cb-9a4b276473c4-kube-api-access-zt5mt\") pod \"f100a752-adf7-4593-b9cb-9a4b276473c4\" (UID: \"f100a752-adf7-4593-b9cb-9a4b276473c4\") "
Oct 30 22:11:49.269789 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:49.269655    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/f100a752-adf7-4593-b9cb-9a4b276473c4-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f100a752-adf7-4593-b9cb-9a4b276473c4-istio-podinfo\") pod \"f100a752-adf7-4593-b9cb-9a4b276473c4\" (UID: \"f100a752-adf7-4593-b9cb-9a4b276473c4\") "
Oct 30 22:11:49.270033 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:49.269874    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df9bbdb1-5a45-4223-b03a-5d5e11542e60-workload-socket" pod "df9bbdb1-5a45-4223-b03a-5d5e11542e60" (UID: "df9bbdb1-5a45-4223-b03a-5d5e11542e60"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:11:49.270454 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:49.270403    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df9bbdb1-5a45-4223-b03a-5d5e11542e60-credential-socket" pod "df9bbdb1-5a45-4223-b03a-5d5e11542e60" (UID: "df9bbdb1-5a45-4223-b03a-5d5e11542e60"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:11:49.270564 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:49.270543    4473 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2jgnc\" (UniqueName: \"kubernetes.io/projected/3c982f07-a1d8-4b18-a065-4e53f7e5022a-kube-api-access-2jgnc\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:11:49.270629 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:49.270568    4473 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/df9bbdb1-5a45-4223-b03a-5d5e11542e60-istiod-ca-cert\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:11:49.270629 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:49.270583    4473 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f100a752-adf7-4593-b9cb-9a4b276473c4-workload-certs\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:11:49.270629 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:49.270601    4473 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3c982f07-a1d8-4b18-a065-4e53f7e5022a-istio-token\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:11:49.270629 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:49.270615    4473 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3c982f07-a1d8-4b18-a065-4e53f7e5022a-workload-certs\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:11:49.270629 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:49.270617    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f100a752-adf7-4593-b9cb-9a4b276473c4-istio-data" pod "f100a752-adf7-4593-b9cb-9a4b276473c4" (UID: "f100a752-adf7-4593-b9cb-9a4b276473c4"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:11:49.270891 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:49.270648    4473 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/df9bbdb1-5a45-4223-b03a-5d5e11542e60-workload-socket\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:11:49.270891 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:49.270663    4473 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/df9bbdb1-5a45-4223-b03a-5d5e11542e60-credential-socket\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:11:49.270891 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:49.270676    4473 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3c982f07-a1d8-4b18-a065-4e53f7e5022a-istio-podinfo\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:11:49.271473 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:49.271443    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df9bbdb1-5a45-4223-b03a-5d5e11542e60-istio-data" pod "df9bbdb1-5a45-4223-b03a-5d5e11542e60" (UID: "df9bbdb1-5a45-4223-b03a-5d5e11542e60"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:11:49.271725 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:49.271697    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f100a752-adf7-4593-b9cb-9a4b276473c4-workload-socket" pod "f100a752-adf7-4593-b9cb-9a4b276473c4" (UID: "f100a752-adf7-4593-b9cb-9a4b276473c4"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:11:49.272021 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:49.271995    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df9bbdb1-5a45-4223-b03a-5d5e11542e60-workload-certs" pod "df9bbdb1-5a45-4223-b03a-5d5e11542e60" (UID: "df9bbdb1-5a45-4223-b03a-5d5e11542e60"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:11:49.272210 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:49.272184    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f100a752-adf7-4593-b9cb-9a4b276473c4-credential-socket" pod "f100a752-adf7-4593-b9cb-9a4b276473c4" (UID: "f100a752-adf7-4593-b9cb-9a4b276473c4"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:11:49.272493 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:49.272469    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f100a752-adf7-4593-b9cb-9a4b276473c4-istiod-ca-cert" pod "f100a752-adf7-4593-b9cb-9a4b276473c4" (UID: "f100a752-adf7-4593-b9cb-9a4b276473c4"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:11:49.273197 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:49.273171    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f100a752-adf7-4593-b9cb-9a4b276473c4-istio-envoy" pod "f100a752-adf7-4593-b9cb-9a4b276473c4" (UID: "f100a752-adf7-4593-b9cb-9a4b276473c4"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:11:49.274504 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:49.274476    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df9bbdb1-5a45-4223-b03a-5d5e11542e60-istio-envoy" pod "df9bbdb1-5a45-4223-b03a-5d5e11542e60" (UID: "df9bbdb1-5a45-4223-b03a-5d5e11542e60"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:11:49.275071 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:49.275044    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/df9bbdb1-5a45-4223-b03a-5d5e11542e60-istio-token" pod "df9bbdb1-5a45-4223-b03a-5d5e11542e60" (UID: "df9bbdb1-5a45-4223-b03a-5d5e11542e60"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:11:49.276773 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:49.276744    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f100a752-adf7-4593-b9cb-9a4b276473c4-istio-podinfo" pod "f100a752-adf7-4593-b9cb-9a4b276473c4" (UID: "f100a752-adf7-4593-b9cb-9a4b276473c4"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:11:49.276885 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:49.276867    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/df9bbdb1-5a45-4223-b03a-5d5e11542e60-kube-api-access-t9px6" pod "df9bbdb1-5a45-4223-b03a-5d5e11542e60" (UID: "df9bbdb1-5a45-4223-b03a-5d5e11542e60"). InnerVolumeSpecName "kube-api-access-t9px6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:11:49.277068 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:49.277049    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f100a752-adf7-4593-b9cb-9a4b276473c4-kube-api-access-zt5mt" pod "f100a752-adf7-4593-b9cb-9a4b276473c4" (UID: "f100a752-adf7-4593-b9cb-9a4b276473c4"). InnerVolumeSpecName "kube-api-access-zt5mt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:11:49.277129 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:49.277076    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f100a752-adf7-4593-b9cb-9a4b276473c4-istio-token" pod "f100a752-adf7-4593-b9cb-9a4b276473c4" (UID: "f100a752-adf7-4593-b9cb-9a4b276473c4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:11:49.279203 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:49.279178    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/df9bbdb1-5a45-4223-b03a-5d5e11542e60-istio-podinfo" pod "df9bbdb1-5a45-4223-b03a-5d5e11542e60" (UID: "df9bbdb1-5a45-4223-b03a-5d5e11542e60"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:11:49.371912 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:49.371870    4473 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/df9bbdb1-5a45-4223-b03a-5d5e11542e60-istio-podinfo\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:11:49.371912 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:49.371901    4473 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f100a752-adf7-4593-b9cb-9a4b276473c4-credential-socket\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:11:49.371912 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:49.371912    4473 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/df9bbdb1-5a45-4223-b03a-5d5e11542e60-istio-envoy\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:11:49.371912 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:49.371936    4473 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f100a752-adf7-4593-b9cb-9a4b276473c4-istio-data\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:11:49.372281 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:49.371948    4473 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f100a752-adf7-4593-b9cb-9a4b276473c4-istio-token\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:11:49.372281 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:49.371961    4473 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f100a752-adf7-4593-b9cb-9a4b276473c4-istiod-ca-cert\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:11:49.372281 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:49.371974    4473 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/df9bbdb1-5a45-4223-b03a-5d5e11542e60-istio-token\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:11:49.372281 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:49.371989    4473 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f100a752-adf7-4593-b9cb-9a4b276473c4-workload-socket\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:11:49.372281 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:49.372002    4473 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/df9bbdb1-5a45-4223-b03a-5d5e11542e60-istio-data\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:11:49.372281 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:49.372012    4473 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/df9bbdb1-5a45-4223-b03a-5d5e11542e60-workload-certs\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:11:49.372281 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:49.372021    4473 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zt5mt\" (UniqueName: \"kubernetes.io/projected/f100a752-adf7-4593-b9cb-9a4b276473c4-kube-api-access-zt5mt\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:11:49.372281 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:49.372031    4473 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f100a752-adf7-4593-b9cb-9a4b276473c4-istio-podinfo\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:11:49.372281 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:49.372039    4473 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f100a752-adf7-4593-b9cb-9a4b276473c4-istio-envoy\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:11:49.372281 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:49.372048    4473 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t9px6\" (UniqueName: \"kubernetes.io/projected/df9bbdb1-5a45-4223-b03a-5d5e11542e60-kube-api-access-t9px6\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:11:49.903453 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:49.903421    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-qng2w"
Oct 30 22:11:49.903962 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:49.903470    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-qng2w" event={"ID":"6560c1bc-b507-4a92-8586-23e0fcc5362a","Type":"ContainerDied","Data":"443f61aee6536dfd73b6485479a5e29f8fa239efcb32f4fac72c9c8086b5ed28"}
Oct 30 22:11:49.903962 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:49.903536    4473 scope.go:122] "RemoveContainer" containerID="d26e70139b180333d21d0d8011cf38c246c5461712eb9266e0a20acae0653eae"
Oct 30 22:11:49.905509 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:49.905489    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-jsdrm"
Oct 30 22:11:49.905639 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:49.905541    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-jsdrm" event={"ID":"f100a752-adf7-4593-b9cb-9a4b276473c4","Type":"ContainerDied","Data":"ccd00b8f71de8c9884d7a0130c8808b8cd6dcbbc80343b962b47693336cd5690"}
Oct 30 22:11:49.907484 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:49.907465    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-ktzwz"
Oct 30 22:11:49.907611 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:49.907466    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-ktzwz" event={"ID":"df9bbdb1-5a45-4223-b03a-5d5e11542e60","Type":"ContainerDied","Data":"71ee30e79eb3ff3e298888b70da5af551ad470cef89edb23c63afaae6b527b8f"}
Oct 30 22:11:49.909700 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:49.909669    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-gsc42" event={"ID":"3c982f07-a1d8-4b18-a065-4e53f7e5022a","Type":"ContainerDied","Data":"89d95c9ebf5a9d57f060a0f595dc5eef3f15774551d43c4ac62550e7a4d6980a"}
Oct 30 22:11:49.909802 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:49.909677    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-gsc42"
Oct 30 22:11:49.918249 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:49.918227    4473 scope.go:122] "RemoveContainer" containerID="de4d576a5ae5e7ccba6f1ff4d4c50cb825c54004dc84421151be2927ad215739"
Oct 30 22:11:49.938019 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:49.934491    4473 scope.go:122] "RemoveContainer" containerID="db07e6e36250d9ebf8ae105051d349e432a4d0ba0d0eaecfe66b14d4a53c972c"
Oct 30 22:11:49.966550 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:49.966525    4473 scope.go:122] "RemoveContainer" containerID="ce6d303a42496486091e3e9f8d8f2e2895ecdcbfe30841234474479823fceed3"
Oct 30 22:11:50.164270 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:50.164190    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-jsdrm"]
Oct 30 22:11:50.172644 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:50.172603    4473 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-jsdrm"]
Oct 30 22:11:50.290104 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:50.290075    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-ktzwz"]
Oct 30 22:11:50.300606 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:50.300576    4473 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-ktzwz"]
Oct 30 22:11:50.381958 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:11:50.381343    4473 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fncvg" containerName="extractor"
Oct 30 22:11:50.385861 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:50.385836    4473 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="df9bbdb1-5a45-4223-b03a-5d5e11542e60" path="/var/lib/kubelet/pods/df9bbdb1-5a45-4223-b03a-5d5e11542e60/volumes"
Oct 30 22:11:50.386571 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:50.386555    4473 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f100a752-adf7-4593-b9cb-9a4b276473c4" path="/var/lib/kubelet/pods/f100a752-adf7-4593-b9cb-9a4b276473c4/volumes"
Oct 30 22:11:50.480764 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:50.480345    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-gsc42"]
Oct 30 22:11:50.484852 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:50.484820    4473 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-gsc42"]
Oct 30 22:11:50.581283 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:50.581204    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-qng2w"]
Oct 30 22:11:50.586600 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:50.586570    4473 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-0c99b1-openshift-default-967855977-qng2w"]
Oct 30 22:11:52.443224 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:52.443178    4473 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3c982f07-a1d8-4b18-a065-4e53f7e5022a" path="/var/lib/kubelet/pods/3c982f07-a1d8-4b18-a065-4e53f7e5022a/volumes"
Oct 30 22:11:52.453967 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:52.453901    4473 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6560c1bc-b507-4a92-8586-23e0fcc5362a" path="/var/lib/kubelet/pods/6560c1bc-b507-4a92-8586-23e0fcc5362a/volumes"
Oct 30 22:11:59.823083 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:59.823040    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-8cv5d"]
Oct 30 22:11:59.824558 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:59.824521    4473 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-8cv5d" podUID="85b6bb64-5545-4a61-bb0d-a4851348fe30" containerName="istio-proxy" containerID="cri-o://9353c1fa5252a41781bb2c79d223f7f527a1e245af4c585422c54ef6382fa335" gracePeriod=30
Oct 30 22:11:59.859520 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:59.859480    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-wrxqs"]
Oct 30 22:11:59.859766 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:59.859739    4473 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-wrxqs" podUID="370d0319-cf46-40d6-8c55-e4fa23e33dd3" containerName="istio-proxy" containerID="cri-o://9b945b7ecd947aae012a9611e1357fb1c653c55a4b5206e9ea6c92930624a0a2" gracePeriod=30
Oct 30 22:11:59.864230 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:59.864182    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-l8zrp"]
Oct 30 22:11:59.864452 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:59.864411    4473 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-l8zrp" podUID="78f06b05-32bf-46bc-bd95-2b3a8da72b5b" containerName="istio-proxy" containerID="cri-o://e0790aefea43f190a602b7d901a99e2d06b68a40b7c1989cc0978c53b99a57d1" gracePeriod=30
Oct 30 22:11:59.909854 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:59.909815    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-j5tqr"]
Oct 30 22:11:59.910114 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:11:59.910089    4473 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-j5tqr" podUID="0f0fbc43-d98b-4b7c-872b-7978f76650ef" containerName="istio-proxy" containerID="cri-o://93f6fe2bc6310bf844bba8d8e01162b501bb6d2f275f229926ca00321452255c" gracePeriod=30
Oct 30 22:12:00.061341 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:00.061311    4473 generic.go:356] "Generic (PLEG): container finished" podID="feeeb03e-f851-4d1e-bcff-0a934226a2d1" containerID="271a8613be6a58fa1d39fb6218ab6cb2dd30ddfb473d04c6bc8618966a2f20bd" exitCode=0
Oct 30 22:12:00.061523 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:00.061396    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w6-5d36f6-openshift-default-555f9495c9-9lsxw" event={"ID":"feeeb03e-f851-4d1e-bcff-0a934226a2d1","Type":"ContainerDied","Data":"271a8613be6a58fa1d39fb6218ab6cb2dd30ddfb473d04c6bc8618966a2f20bd"}
Oct 30 22:12:00.061886 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:00.061871    4473 scope.go:122] "RemoveContainer" containerID="271a8613be6a58fa1d39fb6218ab6cb2dd30ddfb473d04c6bc8618966a2f20bd"
Oct 30 22:12:00.063676 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:00.063637    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:12:00.063815 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:00.063731    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:12:00.063815 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:00.063781    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:12:00.533845 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:00.533812    4473 patch_prober.go:29] interesting pod/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-8cv5d container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 22:12:00.534884 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:00.533868    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-8cv5d" podUID="85b6bb64-5545-4a61-bb0d-a4851348fe30" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 22:12:01.070658 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:01.070627    4473 generic.go:356] "Generic (PLEG): container finished" podID="3e4b6a7f-8bfd-49d8-be83-674d091c99d4" containerID="99754613fc190c15476ba88cb6724abf21129e7aa43928ceaee60e7bc615d280" exitCode=0
Oct 30 22:12:01.071098 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:01.070703    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-8f1d78db-openshift-default-6bcc897b9-r5blj" event={"ID":"3e4b6a7f-8bfd-49d8-be83-674d091c99d4","Type":"ContainerDied","Data":"99754613fc190c15476ba88cb6724abf21129e7aa43928ceaee60e7bc615d280"}
Oct 30 22:12:01.071339 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:01.071319    4473 scope.go:122] "RemoveContainer" containerID="99754613fc190c15476ba88cb6724abf21129e7aa43928ceaee60e7bc615d280"
Oct 30 22:12:01.073195 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:01.073159    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w6-5d36f6-openshift-default-555f9495c9-9lsxw" event={"ID":"feeeb03e-f851-4d1e-bcff-0a934226a2d1","Type":"ContainerStarted","Data":"78c8b90febdf5074794c6910486d9a38965db527e48121b57de16c57225405b6"}
Oct 30 22:12:01.073355 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:01.073326    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:12:01.073431 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:01.073401    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:12:01.073497 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:01.073432    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:12:01.213646 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:01.213609    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w6-5d36f6-openshift-default-555f9495c9-9lsxw"
Oct 30 22:12:01.214106 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:01.214071    4473 patch_prober.go:29] interesting pod/e2e-shared-a-w6-5d36f6-openshift-default-555f9495c9-9lsxw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.112:15021/healthz/ready\": dial tcp 10.133.0.112:15021: connect: connection refused" start-of-body=
Oct 30 22:12:01.214260 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:01.214129    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w6-5d36f6-openshift-default-555f9495c9-9lsxw" podUID="feeeb03e-f851-4d1e-bcff-0a934226a2d1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.112:15021/healthz/ready\": dial tcp 10.133.0.112:15021: connect: connection refused"
Oct 30 22:12:02.085106 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:02.085063    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-8f1d78db-openshift-default-6bcc897b9-r5blj" event={"ID":"3e4b6a7f-8bfd-49d8-be83-674d091c99d4","Type":"ContainerStarted","Data":"3efab88c68c5ee507793ab011d2598e5419f1bed066fba036f9cffd091450550"}
Oct 30 22:12:02.116504 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:02.116465    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-8f1d78db-openshift-default-6bcc897b9-r5blj"
Oct 30 22:12:02.118426 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:02.118393    4473 patch_prober.go:29] interesting pod/e2e-mt-8f1d78db-openshift-default-6bcc897b9-r5blj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.113:15021/healthz/ready\": dial tcp 10.133.0.113:15021: connect: connection refused" start-of-body=
Oct 30 22:12:02.118574 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:02.118451    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-8f1d78db-openshift-default-6bcc897b9-r5blj" podUID="3e4b6a7f-8bfd-49d8-be83-674d091c99d4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.113:15021/healthz/ready\": dial tcp 10.133.0.113:15021: connect: connection refused"
Oct 30 22:12:02.214708 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:02.214674    4473 patch_prober.go:29] interesting pod/e2e-shared-a-w6-5d36f6-openshift-default-555f9495c9-9lsxw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.112:15021/healthz/ready\": dial tcp 10.133.0.112:15021: connect: connection refused" start-of-body=
Oct 30 22:12:02.214883 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:02.214725    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w6-5d36f6-openshift-default-555f9495c9-9lsxw" podUID="feeeb03e-f851-4d1e-bcff-0a934226a2d1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.112:15021/healthz/ready\": dial tcp 10.133.0.112:15021: connect: connection refused"
Oct 30 22:12:03.117323 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:03.117288    4473 patch_prober.go:29] interesting pod/e2e-mt-8f1d78db-openshift-default-6bcc897b9-r5blj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.113:15021/healthz/ready\": dial tcp 10.133.0.113:15021: connect: connection refused" start-of-body=
Oct 30 22:12:03.117814 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:03.117363    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-8f1d78db-openshift-default-6bcc897b9-r5blj" podUID="3e4b6a7f-8bfd-49d8-be83-674d091c99d4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.113:15021/healthz/ready\": dial tcp 10.133.0.113:15021: connect: connection refused"
Oct 30 22:12:03.173953 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:03.173140    4473 patch_prober.go:29] interesting pod/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-wrxqs container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 22:12:03.173953 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:03.173207    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-wrxqs" podUID="370d0319-cf46-40d6-8c55-e4fa23e33dd3" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 22:12:03.214883 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:03.214848    4473 patch_prober.go:29] interesting pod/e2e-shared-a-w6-5d36f6-openshift-default-555f9495c9-9lsxw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.112:15021/healthz/ready\": dial tcp 10.133.0.112:15021: connect: connection refused" start-of-body=
Oct 30 22:12:03.215103 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:03.214959    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w6-5d36f6-openshift-default-555f9495c9-9lsxw" podUID="feeeb03e-f851-4d1e-bcff-0a934226a2d1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.112:15021/healthz/ready\": dial tcp 10.133.0.112:15021: connect: connection refused"
Oct 30 22:12:03.595584 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:03.595486    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-8f1d78db-openshift-default-6bcc897b9-r5blj"
Oct 30 22:12:04.116019 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:04.115966    4473 patch_prober.go:29] interesting pod/e2e-mt-8f1d78db-openshift-default-6bcc897b9-r5blj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.113:15021/healthz/ready\": dial tcp 10.133.0.113:15021: connect: connection refused" start-of-body=
Oct 30 22:12:04.116230 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:04.116036    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-8f1d78db-openshift-default-6bcc897b9-r5blj" podUID="3e4b6a7f-8bfd-49d8-be83-674d091c99d4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.113:15021/healthz/ready\": dial tcp 10.133.0.113:15021: connect: connection refused"
Oct 30 22:12:04.213736 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:04.213693    4473 patch_prober.go:29] interesting pod/e2e-shared-a-w6-5d36f6-openshift-default-555f9495c9-9lsxw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.112:15021/healthz/ready\": dial tcp 10.133.0.112:15021: connect: connection refused" start-of-body=
Oct 30 22:12:04.214183 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:04.213751    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w6-5d36f6-openshift-default-555f9495c9-9lsxw" podUID="feeeb03e-f851-4d1e-bcff-0a934226a2d1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.112:15021/healthz/ready\": dial tcp 10.133.0.112:15021: connect: connection refused"
Oct 30 22:12:04.484228 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:04.484124    4473 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-242ddaf0/e2e-lifecycle-model-242ddaf0-kserve-87686648f-l9tjf"]
Oct 30 22:12:04.485505 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:04.485475    4473 cpu_manager.go:414] "containerMap: removing container" podUID="df9bbdb1-5a45-4223-b03a-5d5e11542e60" containerName="istio-proxy"
Oct 30 22:12:04.485667 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:04.485654    4473 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="df9bbdb1-5a45-4223-b03a-5d5e11542e60" containerName="istio-proxy"
Oct 30 22:12:04.485830 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:04.485816    4473 cpu_manager.go:414] "containerMap: removing container" podUID="6560c1bc-b507-4a92-8586-23e0fcc5362a" containerName="istio-proxy"
Oct 30 22:12:04.485942 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:04.485908    4473 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6560c1bc-b507-4a92-8586-23e0fcc5362a" containerName="istio-proxy"
Oct 30 22:12:04.486076 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:04.486068    4473 cpu_manager.go:414] "containerMap: removing container" podUID="3c982f07-a1d8-4b18-a065-4e53f7e5022a" containerName="istio-proxy"
Oct 30 22:12:04.486129 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:04.486122    4473 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3c982f07-a1d8-4b18-a065-4e53f7e5022a" containerName="istio-proxy"
Oct 30 22:12:04.486176 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:04.486170    4473 cpu_manager.go:414] "containerMap: removing container" podUID="f100a752-adf7-4593-b9cb-9a4b276473c4" containerName="istio-proxy"
Oct 30 22:12:04.486221 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:04.486215    4473 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f100a752-adf7-4593-b9cb-9a4b276473c4" containerName="istio-proxy"
Oct 30 22:12:04.486396 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:04.486388    4473 memory_manager.go:365] "RemoveStaleState removing state" podUID="3c982f07-a1d8-4b18-a065-4e53f7e5022a" containerName="istio-proxy"
Oct 30 22:12:04.486456 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:04.486448    4473 memory_manager.go:365] "RemoveStaleState removing state" podUID="f100a752-adf7-4593-b9cb-9a4b276473c4" containerName="istio-proxy"
Oct 30 22:12:04.486512 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:04.486506    4473 memory_manager.go:365] "RemoveStaleState removing state" podUID="df9bbdb1-5a45-4223-b03a-5d5e11542e60" containerName="istio-proxy"
Oct 30 22:12:04.486561 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:04.486556    4473 memory_manager.go:365] "RemoveStaleState removing state" podUID="6560c1bc-b507-4a92-8586-23e0fcc5362a" containerName="istio-proxy"
Oct 30 22:12:04.489170 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:04.489154    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-242ddaf0/e2e-lifecycle-model-242ddaf0-kserve-87686648f-l9tjf"
Oct 30 22:12:04.512578 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:04.512546    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5fb7086a-96b1-4b1c-a86d-8e38bca4cf80-dshm\") pod \"e2e-lifecycle-model-242ddaf0-kserve-87686648f-l9tjf\" (UID: \"5fb7086a-96b1-4b1c-a86d-8e38bca4cf80\") " pod="ai-tenant-e2e-mt-242ddaf0/e2e-lifecycle-model-242ddaf0-kserve-87686648f-l9tjf"
Oct 30 22:12:04.512785 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:04.512774    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5fb7086a-96b1-4b1c-a86d-8e38bca4cf80-model-cache\") pod \"e2e-lifecycle-model-242ddaf0-kserve-87686648f-l9tjf\" (UID: \"5fb7086a-96b1-4b1c-a86d-8e38bca4cf80\") " pod="ai-tenant-e2e-mt-242ddaf0/e2e-lifecycle-model-242ddaf0-kserve-87686648f-l9tjf"
Oct 30 22:12:04.513013 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:04.513000    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5fb7086a-96b1-4b1c-a86d-8e38bca4cf80-tmp-dir\") pod \"e2e-lifecycle-model-242ddaf0-kserve-87686648f-l9tjf\" (UID: \"5fb7086a-96b1-4b1c-a86d-8e38bca4cf80\") " pod="ai-tenant-e2e-mt-242ddaf0/e2e-lifecycle-model-242ddaf0-kserve-87686648f-l9tjf"
Oct 30 22:12:04.513196 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:04.513184    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d6rht\" (UniqueName: \"kubernetes.io/projected/5fb7086a-96b1-4b1c-a86d-8e38bca4cf80-kube-api-access-d6rht\") pod \"e2e-lifecycle-model-242ddaf0-kserve-87686648f-l9tjf\" (UID: \"5fb7086a-96b1-4b1c-a86d-8e38bca4cf80\") " pod="ai-tenant-e2e-mt-242ddaf0/e2e-lifecycle-model-242ddaf0-kserve-87686648f-l9tjf"
Oct 30 22:12:04.513341 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:04.513331    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5fb7086a-96b1-4b1c-a86d-8e38bca4cf80-home\") pod \"e2e-lifecycle-model-242ddaf0-kserve-87686648f-l9tjf\" (UID: \"5fb7086a-96b1-4b1c-a86d-8e38bca4cf80\") " pod="ai-tenant-e2e-mt-242ddaf0/e2e-lifecycle-model-242ddaf0-kserve-87686648f-l9tjf"
Oct 30 22:12:04.513482 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:04.513470    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5fb7086a-96b1-4b1c-a86d-8e38bca4cf80-tls-certs\") pod \"e2e-lifecycle-model-242ddaf0-kserve-87686648f-l9tjf\" (UID: \"5fb7086a-96b1-4b1c-a86d-8e38bca4cf80\") " pod="ai-tenant-e2e-mt-242ddaf0/e2e-lifecycle-model-242ddaf0-kserve-87686648f-l9tjf"
Oct 30 22:12:04.589106 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:04.582035    4473 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-242ddaf0\"/\"kube-root-ca.crt\""
Oct 30 22:12:04.589106 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:04.582128    4473 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-242ddaf0\"/\"openshift-service-ca.crt\""
Oct 30 22:12:04.631958 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:04.615192    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-242ddaf0\"/\"e2e-lifecycle-model-242ddaf0-kserve-self-signed-certs\""
Oct 30 22:12:04.631958 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:04.615300    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-242ddaf0\"/\"default-dockercfg-v6sfl\""
Oct 30 22:12:04.631958 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:04.616224    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-242ddaf0/e2e-lifecycle-model-242ddaf0-kserve-87686648f-l9tjf"]
Oct 30 22:12:04.631958 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:04.617646    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5fb7086a-96b1-4b1c-a86d-8e38bca4cf80-tls-certs\") pod \"e2e-lifecycle-model-242ddaf0-kserve-87686648f-l9tjf\" (UID: \"5fb7086a-96b1-4b1c-a86d-8e38bca4cf80\") " pod="ai-tenant-e2e-mt-242ddaf0/e2e-lifecycle-model-242ddaf0-kserve-87686648f-l9tjf"
Oct 30 22:12:04.631958 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:04.618396    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5fb7086a-96b1-4b1c-a86d-8e38bca4cf80-dshm\") pod \"e2e-lifecycle-model-242ddaf0-kserve-87686648f-l9tjf\" (UID: \"5fb7086a-96b1-4b1c-a86d-8e38bca4cf80\") " pod="ai-tenant-e2e-mt-242ddaf0/e2e-lifecycle-model-242ddaf0-kserve-87686648f-l9tjf"
Oct 30 22:12:04.631958 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:04.618828    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5fb7086a-96b1-4b1c-a86d-8e38bca4cf80-model-cache\") pod \"e2e-lifecycle-model-242ddaf0-kserve-87686648f-l9tjf\" (UID: \"5fb7086a-96b1-4b1c-a86d-8e38bca4cf80\") " pod="ai-tenant-e2e-mt-242ddaf0/e2e-lifecycle-model-242ddaf0-kserve-87686648f-l9tjf"
Oct 30 22:12:04.631958 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:04.619239    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5fb7086a-96b1-4b1c-a86d-8e38bca4cf80-model-cache\") pod \"e2e-lifecycle-model-242ddaf0-kserve-87686648f-l9tjf\" (UID: \"5fb7086a-96b1-4b1c-a86d-8e38bca4cf80\") " pod="ai-tenant-e2e-mt-242ddaf0/e2e-lifecycle-model-242ddaf0-kserve-87686648f-l9tjf"
Oct 30 22:12:04.631958 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:04.619370    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5fb7086a-96b1-4b1c-a86d-8e38bca4cf80-tmp-dir\") pod \"e2e-lifecycle-model-242ddaf0-kserve-87686648f-l9tjf\" (UID: \"5fb7086a-96b1-4b1c-a86d-8e38bca4cf80\") " pod="ai-tenant-e2e-mt-242ddaf0/e2e-lifecycle-model-242ddaf0-kserve-87686648f-l9tjf"
Oct 30 22:12:04.631958 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:04.619655    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5fb7086a-96b1-4b1c-a86d-8e38bca4cf80-tmp-dir\") pod \"e2e-lifecycle-model-242ddaf0-kserve-87686648f-l9tjf\" (UID: \"5fb7086a-96b1-4b1c-a86d-8e38bca4cf80\") " pod="ai-tenant-e2e-mt-242ddaf0/e2e-lifecycle-model-242ddaf0-kserve-87686648f-l9tjf"
Oct 30 22:12:04.631958 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:04.619814    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d6rht\" (UniqueName: \"kubernetes.io/projected/5fb7086a-96b1-4b1c-a86d-8e38bca4cf80-kube-api-access-d6rht\") pod \"e2e-lifecycle-model-242ddaf0-kserve-87686648f-l9tjf\" (UID: \"5fb7086a-96b1-4b1c-a86d-8e38bca4cf80\") " pod="ai-tenant-e2e-mt-242ddaf0/e2e-lifecycle-model-242ddaf0-kserve-87686648f-l9tjf"
Oct 30 22:12:04.631958 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:04.620270    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5fb7086a-96b1-4b1c-a86d-8e38bca4cf80-home\") pod \"e2e-lifecycle-model-242ddaf0-kserve-87686648f-l9tjf\" (UID: \"5fb7086a-96b1-4b1c-a86d-8e38bca4cf80\") " pod="ai-tenant-e2e-mt-242ddaf0/e2e-lifecycle-model-242ddaf0-kserve-87686648f-l9tjf"
Oct 30 22:12:04.631958 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:04.620754    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5fb7086a-96b1-4b1c-a86d-8e38bca4cf80-home\") pod \"e2e-lifecycle-model-242ddaf0-kserve-87686648f-l9tjf\" (UID: \"5fb7086a-96b1-4b1c-a86d-8e38bca4cf80\") " pod="ai-tenant-e2e-mt-242ddaf0/e2e-lifecycle-model-242ddaf0-kserve-87686648f-l9tjf"
Oct 30 22:12:04.676244 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:04.674968    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5fb7086a-96b1-4b1c-a86d-8e38bca4cf80-tls-certs\") pod \"e2e-lifecycle-model-242ddaf0-kserve-87686648f-l9tjf\" (UID: \"5fb7086a-96b1-4b1c-a86d-8e38bca4cf80\") " pod="ai-tenant-e2e-mt-242ddaf0/e2e-lifecycle-model-242ddaf0-kserve-87686648f-l9tjf"
Oct 30 22:12:04.707039 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:04.707001    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5fb7086a-96b1-4b1c-a86d-8e38bca4cf80-dshm\") pod \"e2e-lifecycle-model-242ddaf0-kserve-87686648f-l9tjf\" (UID: \"5fb7086a-96b1-4b1c-a86d-8e38bca4cf80\") " pod="ai-tenant-e2e-mt-242ddaf0/e2e-lifecycle-model-242ddaf0-kserve-87686648f-l9tjf"
Oct 30 22:12:04.841000 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:04.824128    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d6rht\" (UniqueName: \"kubernetes.io/projected/5fb7086a-96b1-4b1c-a86d-8e38bca4cf80-kube-api-access-d6rht\") pod \"e2e-lifecycle-model-242ddaf0-kserve-87686648f-l9tjf\" (UID: \"5fb7086a-96b1-4b1c-a86d-8e38bca4cf80\") " pod="ai-tenant-e2e-mt-242ddaf0/e2e-lifecycle-model-242ddaf0-kserve-87686648f-l9tjf"
Oct 30 22:12:04.903951 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:04.890510    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-242ddaf0/e2e-lifecycle-model-242ddaf0-kserve-87686648f-l9tjf"
Oct 30 22:12:05.124672 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:05.124639    4473 patch_prober.go:29] interesting pod/e2e-mt-8f1d78db-openshift-default-6bcc897b9-r5blj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.113:15021/healthz/ready\": dial tcp 10.133.0.113:15021: connect: connection refused" start-of-body=
Oct 30 22:12:05.124971 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:05.124938    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-8f1d78db-openshift-default-6bcc897b9-r5blj" podUID="3e4b6a7f-8bfd-49d8-be83-674d091c99d4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.113:15021/healthz/ready\": dial tcp 10.133.0.113:15021: connect: connection refused"
Oct 30 22:12:05.214345 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:05.214118    4473 patch_prober.go:29] interesting pod/e2e-shared-a-w6-5d36f6-openshift-default-555f9495c9-9lsxw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.112:15021/healthz/ready\": dial tcp 10.133.0.112:15021: connect: connection refused" start-of-body=
Oct 30 22:12:05.214345 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:05.214174    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w6-5d36f6-openshift-default-555f9495c9-9lsxw" podUID="feeeb03e-f851-4d1e-bcff-0a934226a2d1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.112:15021/healthz/ready\": dial tcp 10.133.0.112:15021: connect: connection refused"
Oct 30 22:12:05.219168 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:05.219121    4473 generic.go:356] "Generic (PLEG): container finished" podID="370d0319-cf46-40d6-8c55-e4fa23e33dd3" containerID="9b945b7ecd947aae012a9611e1357fb1c653c55a4b5206e9ea6c92930624a0a2" exitCode=0
Oct 30 22:12:05.219487 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:05.219359    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-wrxqs" event={"ID":"370d0319-cf46-40d6-8c55-e4fa23e33dd3","Type":"ContainerDied","Data":"9b945b7ecd947aae012a9611e1357fb1c653c55a4b5206e9ea6c92930624a0a2"}
Oct 30 22:12:05.222713 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:05.222195    4473 generic.go:356] "Generic (PLEG): container finished" podID="78f06b05-32bf-46bc-bd95-2b3a8da72b5b" containerID="e0790aefea43f190a602b7d901a99e2d06b68a40b7c1989cc0978c53b99a57d1" exitCode=0
Oct 30 22:12:05.222713 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:05.222487    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-l8zrp" event={"ID":"78f06b05-32bf-46bc-bd95-2b3a8da72b5b","Type":"ContainerDied","Data":"e0790aefea43f190a602b7d901a99e2d06b68a40b7c1989cc0978c53b99a57d1"}
Oct 30 22:12:05.311165 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:05.311032    4473 generic.go:356] "Generic (PLEG): container finished" podID="85b6bb64-5545-4a61-bb0d-a4851348fe30" containerID="9353c1fa5252a41781bb2c79d223f7f527a1e245af4c585422c54ef6382fa335" exitCode=0
Oct 30 22:12:05.311165 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:05.311098    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-8cv5d" event={"ID":"85b6bb64-5545-4a61-bb0d-a4851348fe30","Type":"ContainerDied","Data":"9353c1fa5252a41781bb2c79d223f7f527a1e245af4c585422c54ef6382fa335"}
Oct 30 22:12:05.494106 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:05.493707    4473 patch_prober.go:29] interesting pod/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-j5tqr container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body=
Oct 30 22:12:05.494106 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:05.493765    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-j5tqr" podUID="0f0fbc43-d98b-4b7c-872b-7978f76650ef" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused"
Oct 30 22:12:05.684445 ip-10-0-140-18 kubenswrapper[4473]: W1030 22:12:05.684408    4473 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5fb7086a_96b1_4b1c_a86d_8e38bca4cf80.slice/crio-c90baa8711af800734967126be53bb571c78e4be042fb3eaa7f5e3ec9cf6e5d0 WatchSource:0}: Error finding container c90baa8711af800734967126be53bb571c78e4be042fb3eaa7f5e3ec9cf6e5d0: Status 404 returned error can't find the container with id c90baa8711af800734967126be53bb571c78e4be042fb3eaa7f5e3ec9cf6e5d0
Oct 30 22:12:05.685568 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:05.685523    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-242ddaf0/e2e-lifecycle-model-242ddaf0-kserve-87686648f-l9tjf"]
Oct 30 22:12:06.101571 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:06.101542    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-l8zrp"
Oct 30 22:12:06.103040 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:06.103016    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-j5tqr"
Oct 30 22:12:06.105989 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:06.105968    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-8cv5d"
Oct 30 22:12:06.117748 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:06.117720    4473 patch_prober.go:29] interesting pod/e2e-mt-8f1d78db-openshift-default-6bcc897b9-r5blj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.113:15021/healthz/ready\": dial tcp 10.133.0.113:15021: connect: connection refused" start-of-body=
Oct 30 22:12:06.117875 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:06.117771    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-8f1d78db-openshift-default-6bcc897b9-r5blj" podUID="3e4b6a7f-8bfd-49d8-be83-674d091c99d4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.113:15021/healthz/ready\": dial tcp 10.133.0.113:15021: connect: connection refused"
Oct 30 22:12:06.118618 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:06.118598    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-wrxqs"
Oct 30 22:12:06.161963 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:06.151052    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/78f06b05-32bf-46bc-bd95-2b3a8da72b5b-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/78f06b05-32bf-46bc-bd95-2b3a8da72b5b-credential-socket\") pod \"78f06b05-32bf-46bc-bd95-2b3a8da72b5b\" (UID: \"78f06b05-32bf-46bc-bd95-2b3a8da72b5b\") "
Oct 30 22:12:06.162642 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:06.162615    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0f0fbc43-d98b-4b7c-872b-7978f76650ef-kube-api-access-bd2bj\" (UniqueName: \"kubernetes.io/projected/0f0fbc43-d98b-4b7c-872b-7978f76650ef-kube-api-access-bd2bj\") pod \"0f0fbc43-d98b-4b7c-872b-7978f76650ef\" (UID: \"0f0fbc43-d98b-4b7c-872b-7978f76650ef\") "
Oct 30 22:12:06.163185 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:06.163165    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0f0fbc43-d98b-4b7c-872b-7978f76650ef-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0f0fbc43-d98b-4b7c-872b-7978f76650ef-workload-socket\") pod \"0f0fbc43-d98b-4b7c-872b-7978f76650ef\" (UID: \"0f0fbc43-d98b-4b7c-872b-7978f76650ef\") "
Oct 30 22:12:06.182947 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:06.170141    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/85b6bb64-5545-4a61-bb0d-a4851348fe30-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/85b6bb64-5545-4a61-bb0d-a4851348fe30-workload-certs\") pod \"85b6bb64-5545-4a61-bb0d-a4851348fe30\" (UID: \"85b6bb64-5545-4a61-bb0d-a4851348fe30\") "
Oct 30 22:12:06.183217 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:06.183191    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0f0fbc43-d98b-4b7c-872b-7978f76650ef-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0f0fbc43-d98b-4b7c-872b-7978f76650ef-workload-certs\") pod \"0f0fbc43-d98b-4b7c-872b-7978f76650ef\" (UID: \"0f0fbc43-d98b-4b7c-872b-7978f76650ef\") "
Oct 30 22:12:06.183392 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:06.183373    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/78f06b05-32bf-46bc-bd95-2b3a8da72b5b-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/78f06b05-32bf-46bc-bd95-2b3a8da72b5b-istio-podinfo\") pod \"78f06b05-32bf-46bc-bd95-2b3a8da72b5b\" (UID: \"78f06b05-32bf-46bc-bd95-2b3a8da72b5b\") "
Oct 30 22:12:06.183584 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:06.183566    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/78f06b05-32bf-46bc-bd95-2b3a8da72b5b-kube-api-access-jcfzn\" (UniqueName: \"kubernetes.io/projected/78f06b05-32bf-46bc-bd95-2b3a8da72b5b-kube-api-access-jcfzn\") pod \"78f06b05-32bf-46bc-bd95-2b3a8da72b5b\" (UID: \"78f06b05-32bf-46bc-bd95-2b3a8da72b5b\") "
Oct 30 22:12:06.186414 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:06.186387    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/85b6bb64-5545-4a61-bb0d-a4851348fe30-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/85b6bb64-5545-4a61-bb0d-a4851348fe30-istio-envoy\") pod \"85b6bb64-5545-4a61-bb0d-a4851348fe30\" (UID: \"85b6bb64-5545-4a61-bb0d-a4851348fe30\") "
Oct 30 22:12:06.186557 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:06.186543    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0f0fbc43-d98b-4b7c-872b-7978f76650ef-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0f0fbc43-d98b-4b7c-872b-7978f76650ef-istio-envoy\") pod \"0f0fbc43-d98b-4b7c-872b-7978f76650ef\" (UID: \"0f0fbc43-d98b-4b7c-872b-7978f76650ef\") "
Oct 30 22:12:06.186697 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:06.186682    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0f0fbc43-d98b-4b7c-872b-7978f76650ef-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0f0fbc43-d98b-4b7c-872b-7978f76650ef-credential-socket\") pod \"0f0fbc43-d98b-4b7c-872b-7978f76650ef\" (UID: \"0f0fbc43-d98b-4b7c-872b-7978f76650ef\") "
Oct 30 22:12:06.186841 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:06.186828    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0f0fbc43-d98b-4b7c-872b-7978f76650ef-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0f0fbc43-d98b-4b7c-872b-7978f76650ef-istiod-ca-cert\") pod \"0f0fbc43-d98b-4b7c-872b-7978f76650ef\" (UID: \"0f0fbc43-d98b-4b7c-872b-7978f76650ef\") "
Oct 30 22:12:06.186979 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:06.162495    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/78f06b05-32bf-46bc-bd95-2b3a8da72b5b-credential-socket" pod "78f06b05-32bf-46bc-bd95-2b3a8da72b5b" (UID: "78f06b05-32bf-46bc-bd95-2b3a8da72b5b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:12:06.186979 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:06.170028    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0f0fbc43-d98b-4b7c-872b-7978f76650ef-workload-socket" pod "0f0fbc43-d98b-4b7c-872b-7978f76650ef" (UID: "0f0fbc43-d98b-4b7c-872b-7978f76650ef"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:12:06.187113 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:06.172224    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/85b6bb64-5545-4a61-bb0d-a4851348fe30-workload-certs" pod "85b6bb64-5545-4a61-bb0d-a4851348fe30" (UID: "85b6bb64-5545-4a61-bb0d-a4851348fe30"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:12:06.187113 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:06.183906    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0f0fbc43-d98b-4b7c-872b-7978f76650ef-workload-certs" pod "0f0fbc43-d98b-4b7c-872b-7978f76650ef" (UID: "0f0fbc43-d98b-4b7c-872b-7978f76650ef"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:12:06.187113 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:06.186791    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0f0fbc43-d98b-4b7c-872b-7978f76650ef-kube-api-access-bd2bj" pod "0f0fbc43-d98b-4b7c-872b-7978f76650ef" (UID: "0f0fbc43-d98b-4b7c-872b-7978f76650ef"). InnerVolumeSpecName "kube-api-access-bd2bj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:12:06.187113 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:06.186950    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/85b6bb64-5545-4a61-bb0d-a4851348fe30-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/85b6bb64-5545-4a61-bb0d-a4851348fe30-credential-socket\") pod \"85b6bb64-5545-4a61-bb0d-a4851348fe30\" (UID: \"85b6bb64-5545-4a61-bb0d-a4851348fe30\") "
Oct 30 22:12:06.187113 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:06.187102    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/85b6bb64-5545-4a61-bb0d-a4851348fe30-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/85b6bb64-5545-4a61-bb0d-a4851348fe30-istiod-ca-cert\") pod \"85b6bb64-5545-4a61-bb0d-a4851348fe30\" (UID: \"85b6bb64-5545-4a61-bb0d-a4851348fe30\") "
Oct 30 22:12:06.187365 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:06.187154    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/370d0319-cf46-40d6-8c55-e4fa23e33dd3-kube-api-access-ttzs8\" (UniqueName: \"kubernetes.io/projected/370d0319-cf46-40d6-8c55-e4fa23e33dd3-kube-api-access-ttzs8\") pod \"370d0319-cf46-40d6-8c55-e4fa23e33dd3\" (UID: \"370d0319-cf46-40d6-8c55-e4fa23e33dd3\") "
Oct 30 22:12:06.187365 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:06.187181    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/78f06b05-32bf-46bc-bd95-2b3a8da72b5b-istio-token\" (UniqueName: \"kubernetes.io/projected/78f06b05-32bf-46bc-bd95-2b3a8da72b5b-istio-token\") pod \"78f06b05-32bf-46bc-bd95-2b3a8da72b5b\" (UID: \"78f06b05-32bf-46bc-bd95-2b3a8da72b5b\") "
Oct 30 22:12:06.187365 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:06.187240    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/78f06b05-32bf-46bc-bd95-2b3a8da72b5b-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/78f06b05-32bf-46bc-bd95-2b3a8da72b5b-istio-envoy\") pod \"78f06b05-32bf-46bc-bd95-2b3a8da72b5b\" (UID: \"78f06b05-32bf-46bc-bd95-2b3a8da72b5b\") "
Oct 30 22:12:06.187365 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:06.187288    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/78f06b05-32bf-46bc-bd95-2b3a8da72b5b-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/78f06b05-32bf-46bc-bd95-2b3a8da72b5b-workload-socket\") pod \"78f06b05-32bf-46bc-bd95-2b3a8da72b5b\" (UID: \"78f06b05-32bf-46bc-bd95-2b3a8da72b5b\") "
Oct 30 22:12:06.187365 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:06.187309    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0f0fbc43-d98b-4b7c-872b-7978f76650ef-credential-socket" pod "0f0fbc43-d98b-4b7c-872b-7978f76650ef" (UID: "0f0fbc43-d98b-4b7c-872b-7978f76650ef"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:12:06.187365 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:06.187325    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/370d0319-cf46-40d6-8c55-e4fa23e33dd3-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/370d0319-cf46-40d6-8c55-e4fa23e33dd3-istio-envoy\") pod \"370d0319-cf46-40d6-8c55-e4fa23e33dd3\" (UID: \"370d0319-cf46-40d6-8c55-e4fa23e33dd3\") "
Oct 30 22:12:06.187685 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:06.187447    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/78f06b05-32bf-46bc-bd95-2b3a8da72b5b-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/78f06b05-32bf-46bc-bd95-2b3a8da72b5b-istio-data\") pod \"78f06b05-32bf-46bc-bd95-2b3a8da72b5b\" (UID: \"78f06b05-32bf-46bc-bd95-2b3a8da72b5b\") "
Oct 30 22:12:06.187685 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:06.187562    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/370d0319-cf46-40d6-8c55-e4fa23e33dd3-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/370d0319-cf46-40d6-8c55-e4fa23e33dd3-workload-certs\") pod \"370d0319-cf46-40d6-8c55-e4fa23e33dd3\" (UID: \"370d0319-cf46-40d6-8c55-e4fa23e33dd3\") "
Oct 30 22:12:06.187685 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:06.187600    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/370d0319-cf46-40d6-8c55-e4fa23e33dd3-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/370d0319-cf46-40d6-8c55-e4fa23e33dd3-istio-data\") pod \"370d0319-cf46-40d6-8c55-e4fa23e33dd3\" (UID: \"370d0319-cf46-40d6-8c55-e4fa23e33dd3\") "
Oct 30 22:12:06.187685 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:06.187631    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0f0fbc43-d98b-4b7c-872b-7978f76650ef-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0f0fbc43-d98b-4b7c-872b-7978f76650ef-istio-data\") pod \"0f0fbc43-d98b-4b7c-872b-7978f76650ef\" (UID: \"0f0fbc43-d98b-4b7c-872b-7978f76650ef\") "
Oct 30 22:12:06.187685 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:06.187680    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0f0fbc43-d98b-4b7c-872b-7978f76650ef-istio-token\" (UniqueName: \"kubernetes.io/projected/0f0fbc43-d98b-4b7c-872b-7978f76650ef-istio-token\") pod \"0f0fbc43-d98b-4b7c-872b-7978f76650ef\" (UID: \"0f0fbc43-d98b-4b7c-872b-7978f76650ef\") "
Oct 30 22:12:06.187958 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:06.187722    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/0f0fbc43-d98b-4b7c-872b-7978f76650ef-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0f0fbc43-d98b-4b7c-872b-7978f76650ef-istio-podinfo\") pod \"0f0fbc43-d98b-4b7c-872b-7978f76650ef\" (UID: \"0f0fbc43-d98b-4b7c-872b-7978f76650ef\") "
Oct 30 22:12:06.187958 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:06.187754    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/370d0319-cf46-40d6-8c55-e4fa23e33dd3-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/370d0319-cf46-40d6-8c55-e4fa23e33dd3-istio-podinfo\") pod \"370d0319-cf46-40d6-8c55-e4fa23e33dd3\" (UID: \"370d0319-cf46-40d6-8c55-e4fa23e33dd3\") "
Oct 30 22:12:06.187958 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:06.187807    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/370d0319-cf46-40d6-8c55-e4fa23e33dd3-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/370d0319-cf46-40d6-8c55-e4fa23e33dd3-istiod-ca-cert\") pod \"370d0319-cf46-40d6-8c55-e4fa23e33dd3\" (UID: \"370d0319-cf46-40d6-8c55-e4fa23e33dd3\") "
Oct 30 22:12:06.187958 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:06.187878    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/370d0319-cf46-40d6-8c55-e4fa23e33dd3-istio-token\" (UniqueName: \"kubernetes.io/projected/370d0319-cf46-40d6-8c55-e4fa23e33dd3-istio-token\") pod \"370d0319-cf46-40d6-8c55-e4fa23e33dd3\" (UID: \"370d0319-cf46-40d6-8c55-e4fa23e33dd3\") "
Oct 30 22:12:06.187958 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:06.187893    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0f0fbc43-d98b-4b7c-872b-7978f76650ef-istiod-ca-cert" pod "0f0fbc43-d98b-4b7c-872b-7978f76650ef" (UID: "0f0fbc43-d98b-4b7c-872b-7978f76650ef"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:12:06.187958 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:06.187934    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/85b6bb64-5545-4a61-bb0d-a4851348fe30-kube-api-access-d42z4\" (UniqueName: \"kubernetes.io/projected/85b6bb64-5545-4a61-bb0d-a4851348fe30-kube-api-access-d42z4\") pod \"85b6bb64-5545-4a61-bb0d-a4851348fe30\" (UID: \"85b6bb64-5545-4a61-bb0d-a4851348fe30\") "
Oct 30 22:12:06.188315 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:06.187980    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/78f06b05-32bf-46bc-bd95-2b3a8da72b5b-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/78f06b05-32bf-46bc-bd95-2b3a8da72b5b-istiod-ca-cert\") pod \"78f06b05-32bf-46bc-bd95-2b3a8da72b5b\" (UID: \"78f06b05-32bf-46bc-bd95-2b3a8da72b5b\") "
Oct 30 22:12:06.188315 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:06.188032    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/85b6bb64-5545-4a61-bb0d-a4851348fe30-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/85b6bb64-5545-4a61-bb0d-a4851348fe30-istio-data\") pod \"85b6bb64-5545-4a61-bb0d-a4851348fe30\" (UID: \"85b6bb64-5545-4a61-bb0d-a4851348fe30\") "
Oct 30 22:12:06.188315 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:06.188061    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/78f06b05-32bf-46bc-bd95-2b3a8da72b5b-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/78f06b05-32bf-46bc-bd95-2b3a8da72b5b-workload-certs\") pod \"78f06b05-32bf-46bc-bd95-2b3a8da72b5b\" (UID: \"78f06b05-32bf-46bc-bd95-2b3a8da72b5b\") "
Oct 30 22:12:06.188315 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:06.188101    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/85b6bb64-5545-4a61-bb0d-a4851348fe30-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/85b6bb64-5545-4a61-bb0d-a4851348fe30-workload-socket\") pod \"85b6bb64-5545-4a61-bb0d-a4851348fe30\" (UID: \"85b6bb64-5545-4a61-bb0d-a4851348fe30\") "
Oct 30 22:12:06.188315 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:06.188139    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/370d0319-cf46-40d6-8c55-e4fa23e33dd3-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/370d0319-cf46-40d6-8c55-e4fa23e33dd3-credential-socket\") pod \"370d0319-cf46-40d6-8c55-e4fa23e33dd3\" (UID: \"370d0319-cf46-40d6-8c55-e4fa23e33dd3\") "
Oct 30 22:12:06.188315 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:06.188184    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/85b6bb64-5545-4a61-bb0d-a4851348fe30-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/85b6bb64-5545-4a61-bb0d-a4851348fe30-istio-podinfo\") pod \"85b6bb64-5545-4a61-bb0d-a4851348fe30\" (UID: \"85b6bb64-5545-4a61-bb0d-a4851348fe30\") "
Oct 30 22:12:06.188315 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:06.188218    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/85b6bb64-5545-4a61-bb0d-a4851348fe30-istio-token\" (UniqueName: \"kubernetes.io/projected/85b6bb64-5545-4a61-bb0d-a4851348fe30-istio-token\") pod \"85b6bb64-5545-4a61-bb0d-a4851348fe30\" (UID: \"85b6bb64-5545-4a61-bb0d-a4851348fe30\") "
Oct 30 22:12:06.188315 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:06.188225    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/370d0319-cf46-40d6-8c55-e4fa23e33dd3-workload-certs" pod "370d0319-cf46-40d6-8c55-e4fa23e33dd3" (UID: "370d0319-cf46-40d6-8c55-e4fa23e33dd3"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:12:06.188730 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:06.188320    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/370d0319-cf46-40d6-8c55-e4fa23e33dd3-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/370d0319-cf46-40d6-8c55-e4fa23e33dd3-workload-socket\") pod \"370d0319-cf46-40d6-8c55-e4fa23e33dd3\" (UID: \"370d0319-cf46-40d6-8c55-e4fa23e33dd3\") "
Oct 30 22:12:06.188730 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:06.188530    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/85b6bb64-5545-4a61-bb0d-a4851348fe30-istiod-ca-cert" pod "85b6bb64-5545-4a61-bb0d-a4851348fe30" (UID: "85b6bb64-5545-4a61-bb0d-a4851348fe30"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:12:06.188838 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:06.188800    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/370d0319-cf46-40d6-8c55-e4fa23e33dd3-workload-socket" pod "370d0319-cf46-40d6-8c55-e4fa23e33dd3" (UID: "370d0319-cf46-40d6-8c55-e4fa23e33dd3"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:12:06.190832 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:06.189023    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/370d0319-cf46-40d6-8c55-e4fa23e33dd3-istio-data" pod "370d0319-cf46-40d6-8c55-e4fa23e33dd3" (UID: "370d0319-cf46-40d6-8c55-e4fa23e33dd3"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:12:06.190832 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:06.189322    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/85b6bb64-5545-4a61-bb0d-a4851348fe30-istio-envoy" pod "85b6bb64-5545-4a61-bb0d-a4851348fe30" (UID: "85b6bb64-5545-4a61-bb0d-a4851348fe30"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:12:06.190832 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:06.189380    4473 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0f0fbc43-d98b-4b7c-872b-7978f76650ef-credential-socket\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:12:06.190832 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:06.189401    4473 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0f0fbc43-d98b-4b7c-872b-7978f76650ef-istiod-ca-cert\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:12:06.190832 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:06.189417    4473 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/85b6bb64-5545-4a61-bb0d-a4851348fe30-istiod-ca-cert\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:12:06.190832 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:06.189430    4473 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/370d0319-cf46-40d6-8c55-e4fa23e33dd3-workload-certs\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:12:06.190832 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:06.189444    4473 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/370d0319-cf46-40d6-8c55-e4fa23e33dd3-istio-data\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:12:06.190832 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:06.189495    4473 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/370d0319-cf46-40d6-8c55-e4fa23e33dd3-workload-socket\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:12:06.190832 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:06.189550    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/78f06b05-32bf-46bc-bd95-2b3a8da72b5b-workload-socket" pod "78f06b05-32bf-46bc-bd95-2b3a8da72b5b" (UID: "78f06b05-32bf-46bc-bd95-2b3a8da72b5b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:12:06.190832 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:06.189554    4473 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/78f06b05-32bf-46bc-bd95-2b3a8da72b5b-credential-socket\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:12:06.190832 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:06.189680    4473 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bd2bj\" (UniqueName: \"kubernetes.io/projected/0f0fbc43-d98b-4b7c-872b-7978f76650ef-kube-api-access-bd2bj\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:12:06.190832 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:06.189703    4473 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0f0fbc43-d98b-4b7c-872b-7978f76650ef-workload-socket\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:12:06.190832 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:06.189719    4473 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/85b6bb64-5545-4a61-bb0d-a4851348fe30-workload-certs\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:12:06.190832 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:06.189733    4473 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0f0fbc43-d98b-4b7c-872b-7978f76650ef-workload-certs\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:12:06.190832 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:06.190068    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/78f06b05-32bf-46bc-bd95-2b3a8da72b5b-istiod-ca-cert" pod "78f06b05-32bf-46bc-bd95-2b3a8da72b5b" (UID: "78f06b05-32bf-46bc-bd95-2b3a8da72b5b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:12:06.190832 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:06.190304    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/85b6bb64-5545-4a61-bb0d-a4851348fe30-istio-data" pod "85b6bb64-5545-4a61-bb0d-a4851348fe30" (UID: "85b6bb64-5545-4a61-bb0d-a4851348fe30"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:12:06.190832 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:06.190472    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/78f06b05-32bf-46bc-bd95-2b3a8da72b5b-workload-certs" pod "78f06b05-32bf-46bc-bd95-2b3a8da72b5b" (UID: "78f06b05-32bf-46bc-bd95-2b3a8da72b5b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:12:06.190832 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:06.190642    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/85b6bb64-5545-4a61-bb0d-a4851348fe30-workload-socket" pod "85b6bb64-5545-4a61-bb0d-a4851348fe30" (UID: "85b6bb64-5545-4a61-bb0d-a4851348fe30"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:12:06.190832 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:06.190826    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/370d0319-cf46-40d6-8c55-e4fa23e33dd3-credential-socket" pod "370d0319-cf46-40d6-8c55-e4fa23e33dd3" (UID: "370d0319-cf46-40d6-8c55-e4fa23e33dd3"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:12:06.193092 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:06.192078    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/370d0319-cf46-40d6-8c55-e4fa23e33dd3-kube-api-access-ttzs8" pod "370d0319-cf46-40d6-8c55-e4fa23e33dd3" (UID: "370d0319-cf46-40d6-8c55-e4fa23e33dd3"). InnerVolumeSpecName "kube-api-access-ttzs8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:12:06.193092 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:06.192242    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0f0fbc43-d98b-4b7c-872b-7978f76650ef-istio-envoy" pod "0f0fbc43-d98b-4b7c-872b-7978f76650ef" (UID: "0f0fbc43-d98b-4b7c-872b-7978f76650ef"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:12:06.193426 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:06.193406    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/85b6bb64-5545-4a61-bb0d-a4851348fe30-credential-socket" pod "85b6bb64-5545-4a61-bb0d-a4851348fe30" (UID: "85b6bb64-5545-4a61-bb0d-a4851348fe30"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:12:06.193552 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:06.193517    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/85b6bb64-5545-4a61-bb0d-a4851348fe30-istio-podinfo" pod "85b6bb64-5545-4a61-bb0d-a4851348fe30" (UID: "85b6bb64-5545-4a61-bb0d-a4851348fe30"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:12:06.193676 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:06.193657    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/78f06b05-32bf-46bc-bd95-2b3a8da72b5b-kube-api-access-jcfzn" pod "78f06b05-32bf-46bc-bd95-2b3a8da72b5b" (UID: "78f06b05-32bf-46bc-bd95-2b3a8da72b5b"). InnerVolumeSpecName "kube-api-access-jcfzn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:12:06.194022 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:06.194005    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0f0fbc43-d98b-4b7c-872b-7978f76650ef-istio-data" pod "0f0fbc43-d98b-4b7c-872b-7978f76650ef" (UID: "0f0fbc43-d98b-4b7c-872b-7978f76650ef"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:12:06.194270 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:06.194245    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/78f06b05-32bf-46bc-bd95-2b3a8da72b5b-istio-data" pod "78f06b05-32bf-46bc-bd95-2b3a8da72b5b" (UID: "78f06b05-32bf-46bc-bd95-2b3a8da72b5b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:12:06.195137 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:06.194546    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/370d0319-cf46-40d6-8c55-e4fa23e33dd3-istiod-ca-cert" pod "370d0319-cf46-40d6-8c55-e4fa23e33dd3" (UID: "370d0319-cf46-40d6-8c55-e4fa23e33dd3"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:12:06.197354 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:06.197325    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/78f06b05-32bf-46bc-bd95-2b3a8da72b5b-istio-envoy" pod "78f06b05-32bf-46bc-bd95-2b3a8da72b5b" (UID: "78f06b05-32bf-46bc-bd95-2b3a8da72b5b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:12:06.199064 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:06.199023    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/370d0319-cf46-40d6-8c55-e4fa23e33dd3-istio-envoy" pod "370d0319-cf46-40d6-8c55-e4fa23e33dd3" (UID: "370d0319-cf46-40d6-8c55-e4fa23e33dd3"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:12:06.199357 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:06.199329    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/85b6bb64-5545-4a61-bb0d-a4851348fe30-kube-api-access-d42z4" pod "85b6bb64-5545-4a61-bb0d-a4851348fe30" (UID: "85b6bb64-5545-4a61-bb0d-a4851348fe30"). InnerVolumeSpecName "kube-api-access-d42z4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:12:06.199598 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:06.199569    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0f0fbc43-d98b-4b7c-872b-7978f76650ef-istio-token" pod "0f0fbc43-d98b-4b7c-872b-7978f76650ef" (UID: "0f0fbc43-d98b-4b7c-872b-7978f76650ef"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:12:06.199680 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:06.199661    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/78f06b05-32bf-46bc-bd95-2b3a8da72b5b-istio-token" pod "78f06b05-32bf-46bc-bd95-2b3a8da72b5b" (UID: "78f06b05-32bf-46bc-bd95-2b3a8da72b5b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:12:06.199818 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:06.199790    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/78f06b05-32bf-46bc-bd95-2b3a8da72b5b-istio-podinfo" pod "78f06b05-32bf-46bc-bd95-2b3a8da72b5b" (UID: "78f06b05-32bf-46bc-bd95-2b3a8da72b5b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:12:06.200444 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:06.200411    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0f0fbc43-d98b-4b7c-872b-7978f76650ef-istio-podinfo" pod "0f0fbc43-d98b-4b7c-872b-7978f76650ef" (UID: "0f0fbc43-d98b-4b7c-872b-7978f76650ef"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:12:06.200898 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:06.200876    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/370d0319-cf46-40d6-8c55-e4fa23e33dd3-istio-token" pod "370d0319-cf46-40d6-8c55-e4fa23e33dd3" (UID: "370d0319-cf46-40d6-8c55-e4fa23e33dd3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:12:06.202289 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:06.202270    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/85b6bb64-5545-4a61-bb0d-a4851348fe30-istio-token" pod "85b6bb64-5545-4a61-bb0d-a4851348fe30" (UID: "85b6bb64-5545-4a61-bb0d-a4851348fe30"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:12:06.202356 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:06.202282    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/370d0319-cf46-40d6-8c55-e4fa23e33dd3-istio-podinfo" pod "370d0319-cf46-40d6-8c55-e4fa23e33dd3" (UID: "370d0319-cf46-40d6-8c55-e4fa23e33dd3"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:12:06.214833 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:06.214799    4473 patch_prober.go:29] interesting pod/e2e-shared-a-w6-5d36f6-openshift-default-555f9495c9-9lsxw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.112:15021/healthz/ready\": dial tcp 10.133.0.112:15021: connect: connection refused" start-of-body=
Oct 30 22:12:06.215205 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:06.214859    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w6-5d36f6-openshift-default-555f9495c9-9lsxw" podUID="feeeb03e-f851-4d1e-bcff-0a934226a2d1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.112:15021/healthz/ready\": dial tcp 10.133.0.112:15021: connect: connection refused"
Oct 30 22:12:06.292699 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:06.292445    4473 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/85b6bb64-5545-4a61-bb0d-a4851348fe30-credential-socket\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:12:06.292699 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:06.292477    4473 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ttzs8\" (UniqueName: \"kubernetes.io/projected/370d0319-cf46-40d6-8c55-e4fa23e33dd3-kube-api-access-ttzs8\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:12:06.292699 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:06.292488    4473 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/78f06b05-32bf-46bc-bd95-2b3a8da72b5b-istio-token\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:12:06.292699 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:06.292497    4473 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/78f06b05-32bf-46bc-bd95-2b3a8da72b5b-istio-envoy\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:12:06.292699 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:06.292506    4473 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/78f06b05-32bf-46bc-bd95-2b3a8da72b5b-workload-socket\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:12:06.292699 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:06.292517    4473 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/370d0319-cf46-40d6-8c55-e4fa23e33dd3-istio-envoy\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:12:06.292699 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:06.292525    4473 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/78f06b05-32bf-46bc-bd95-2b3a8da72b5b-istio-data\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:12:06.292699 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:06.292534    4473 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0f0fbc43-d98b-4b7c-872b-7978f76650ef-istio-data\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:12:06.292699 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:06.292542    4473 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0f0fbc43-d98b-4b7c-872b-7978f76650ef-istio-token\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:12:06.292699 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:06.292549    4473 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0f0fbc43-d98b-4b7c-872b-7978f76650ef-istio-podinfo\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:12:06.292699 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:06.292557    4473 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/370d0319-cf46-40d6-8c55-e4fa23e33dd3-istio-podinfo\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:12:06.292699 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:06.292567    4473 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/370d0319-cf46-40d6-8c55-e4fa23e33dd3-istiod-ca-cert\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:12:06.292699 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:06.292574    4473 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/370d0319-cf46-40d6-8c55-e4fa23e33dd3-istio-token\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:12:06.292699 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:06.292582    4473 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d42z4\" (UniqueName: \"kubernetes.io/projected/85b6bb64-5545-4a61-bb0d-a4851348fe30-kube-api-access-d42z4\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:12:06.292699 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:06.292591    4473 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/78f06b05-32bf-46bc-bd95-2b3a8da72b5b-istiod-ca-cert\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:12:06.292699 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:06.292599    4473 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/85b6bb64-5545-4a61-bb0d-a4851348fe30-istio-data\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:12:06.292699 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:06.292608    4473 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/78f06b05-32bf-46bc-bd95-2b3a8da72b5b-workload-certs\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:12:06.292699 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:06.292616    4473 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/85b6bb64-5545-4a61-bb0d-a4851348fe30-workload-socket\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:12:06.292699 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:06.292625    4473 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/370d0319-cf46-40d6-8c55-e4fa23e33dd3-credential-socket\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:12:06.292699 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:06.292635    4473 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/85b6bb64-5545-4a61-bb0d-a4851348fe30-istio-podinfo\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:12:06.292699 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:06.292644    4473 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/85b6bb64-5545-4a61-bb0d-a4851348fe30-istio-token\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:12:06.292699 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:06.292652    4473 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/78f06b05-32bf-46bc-bd95-2b3a8da72b5b-istio-podinfo\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:12:06.292699 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:06.292661    4473 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jcfzn\" (UniqueName: \"kubernetes.io/projected/78f06b05-32bf-46bc-bd95-2b3a8da72b5b-kube-api-access-jcfzn\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:12:06.292699 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:06.292669    4473 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/85b6bb64-5545-4a61-bb0d-a4851348fe30-istio-envoy\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:12:06.292699 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:06.292677    4473 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0f0fbc43-d98b-4b7c-872b-7978f76650ef-istio-envoy\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:12:06.325022 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:06.320089    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-wrxqs"
Oct 30 22:12:06.325022 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:06.320093    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-wrxqs" event={"ID":"370d0319-cf46-40d6-8c55-e4fa23e33dd3","Type":"ContainerDied","Data":"322c5185bd2e5730c85ce3c9ac57326674f9bab74410d3adf6d320133f945e91"}
Oct 30 22:12:06.325022 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:06.320218    4473 scope.go:122] "RemoveContainer" containerID="9b945b7ecd947aae012a9611e1357fb1c653c55a4b5206e9ea6c92930624a0a2"
Oct 30 22:12:06.356939 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:06.345296    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-l8zrp" event={"ID":"78f06b05-32bf-46bc-bd95-2b3a8da72b5b","Type":"ContainerDied","Data":"38bc016df66538f05af9ba50bac1b27d4b3fa91fec8d8d81b27218630f17af2a"}
Oct 30 22:12:06.356939 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:06.345416    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-l8zrp"
Oct 30 22:12:06.401818 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:06.401788    4473 scope.go:122] "RemoveContainer" containerID="e0790aefea43f190a602b7d901a99e2d06b68a40b7c1989cc0978c53b99a57d1"
Oct 30 22:12:06.407087 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:06.405161    4473 generic.go:356] "Generic (PLEG): container finished" podID="0f0fbc43-d98b-4b7c-872b-7978f76650ef" containerID="93f6fe2bc6310bf844bba8d8e01162b501bb6d2f275f229926ca00321452255c" exitCode=0
Oct 30 22:12:06.407087 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:06.405339    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-j5tqr"
Oct 30 22:12:06.410840 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:06.410800    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-j5tqr" event={"ID":"0f0fbc43-d98b-4b7c-872b-7978f76650ef","Type":"ContainerDied","Data":"93f6fe2bc6310bf844bba8d8e01162b501bb6d2f275f229926ca00321452255c"}
Oct 30 22:12:06.411016 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:06.410860    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-j5tqr" event={"ID":"0f0fbc43-d98b-4b7c-872b-7978f76650ef","Type":"ContainerDied","Data":"7bf5c3ee7ea91f054f19d55b7bf77d8b53de7326111d8186516c06a50f58be26"}
Oct 30 22:12:06.413889 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:06.413862    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-242ddaf0/e2e-lifecycle-model-242ddaf0-kserve-87686648f-l9tjf" event={"ID":"5fb7086a-96b1-4b1c-a86d-8e38bca4cf80","Type":"ContainerStarted","Data":"88390cd46f489f97c53e8647fbbaf86c6d03b7585c2053bf8237d103a3968a5f"}
Oct 30 22:12:06.414064 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:06.413901    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-242ddaf0/e2e-lifecycle-model-242ddaf0-kserve-87686648f-l9tjf" event={"ID":"5fb7086a-96b1-4b1c-a86d-8e38bca4cf80","Type":"ContainerStarted","Data":"c90baa8711af800734967126be53bb571c78e4be042fb3eaa7f5e3ec9cf6e5d0"}
Oct 30 22:12:06.417410 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:06.417383    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-8cv5d" event={"ID":"85b6bb64-5545-4a61-bb0d-a4851348fe30","Type":"ContainerDied","Data":"7c2e7eef5805861fa6e3643983d3dd6be565b5ca7295b921f3f989976b0c8f54"}
Oct 30 22:12:06.417599 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:06.417578    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-8cv5d"
Oct 30 22:12:06.418111 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:06.418065    4473 scope.go:122] "RemoveContainer" containerID="93f6fe2bc6310bf844bba8d8e01162b501bb6d2f275f229926ca00321452255c"
Oct 30 22:12:06.450153 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:06.450077    4473 scope.go:122] "RemoveContainer" containerID="3b0fe4884f0af7c72480e3c91905a0dd01fe4f95e9c9ba8c7665ad3c1f926e4c"
Oct 30 22:12:06.478206 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:06.478183    4473 scope.go:122] "RemoveContainer" containerID="93f6fe2bc6310bf844bba8d8e01162b501bb6d2f275f229926ca00321452255c"
Oct 30 22:12:06.478475 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:12:06.478445    4473 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"93f6fe2bc6310bf844bba8d8e01162b501bb6d2f275f229926ca00321452255c\": container with ID starting with 93f6fe2bc6310bf844bba8d8e01162b501bb6d2f275f229926ca00321452255c not found: ID does not exist" containerID="93f6fe2bc6310bf844bba8d8e01162b501bb6d2f275f229926ca00321452255c"
Oct 30 22:12:06.478575 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:06.478489    4473 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"93f6fe2bc6310bf844bba8d8e01162b501bb6d2f275f229926ca00321452255c"} err="failed to get container status \"93f6fe2bc6310bf844bba8d8e01162b501bb6d2f275f229926ca00321452255c\": rpc error: code = NotFound desc = could not find container \"93f6fe2bc6310bf844bba8d8e01162b501bb6d2f275f229926ca00321452255c\": container with ID starting with 93f6fe2bc6310bf844bba8d8e01162b501bb6d2f275f229926ca00321452255c not found: ID does not exist"
Oct 30 22:12:06.478575 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:06.478514    4473 scope.go:122] "RemoveContainer" containerID="3b0fe4884f0af7c72480e3c91905a0dd01fe4f95e9c9ba8c7665ad3c1f926e4c"
Oct 30 22:12:06.478861 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:12:06.478845    4473 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3b0fe4884f0af7c72480e3c91905a0dd01fe4f95e9c9ba8c7665ad3c1f926e4c\": container with ID starting with 3b0fe4884f0af7c72480e3c91905a0dd01fe4f95e9c9ba8c7665ad3c1f926e4c not found: ID does not exist" containerID="3b0fe4884f0af7c72480e3c91905a0dd01fe4f95e9c9ba8c7665ad3c1f926e4c"
Oct 30 22:12:06.479020 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:06.478867    4473 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3b0fe4884f0af7c72480e3c91905a0dd01fe4f95e9c9ba8c7665ad3c1f926e4c"} err="failed to get container status \"3b0fe4884f0af7c72480e3c91905a0dd01fe4f95e9c9ba8c7665ad3c1f926e4c\": rpc error: code = NotFound desc = could not find container \"3b0fe4884f0af7c72480e3c91905a0dd01fe4f95e9c9ba8c7665ad3c1f926e4c\": container with ID starting with 3b0fe4884f0af7c72480e3c91905a0dd01fe4f95e9c9ba8c7665ad3c1f926e4c not found: ID does not exist"
Oct 30 22:12:06.479020 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:06.478882    4473 scope.go:122] "RemoveContainer" containerID="9353c1fa5252a41781bb2c79d223f7f527a1e245af4c585422c54ef6382fa335"
Oct 30 22:12:06.785275 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:06.785239    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-l8zrp"]
Oct 30 22:12:06.791644 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:06.791615    4473 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-l8zrp"]
Oct 30 22:12:06.896257 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:06.896231    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-242ddaf0/e2e-lifecycle-model-242ddaf0-kserve-87686648f-l9tjf"
Oct 30 22:12:06.911466 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:06.911386    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-8cv5d"]
Oct 30 22:12:06.923599 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:06.923569    4473 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-8cv5d"]
Oct 30 22:12:07.055174 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:07.055115    4473 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-mt-242ddaf0/e2e-lifecycle-model-242ddaf0-kserve-87686648f-l9tjf" podStartSLOduration=3.05509907 podStartE2EDuration="3.05509907s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:12:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:12:07.049728141 +0000 UTC m=+1772.823096629" watchObservedRunningTime="2026-10-30 22:12:07.05509907 +0000 UTC m=+1772.828467560"
Oct 30 22:12:07.116010 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:07.115970    4473 patch_prober.go:29] interesting pod/e2e-mt-8f1d78db-openshift-default-6bcc897b9-r5blj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.113:15021/healthz/ready\": dial tcp 10.133.0.113:15021: connect: connection refused" start-of-body=
Oct 30 22:12:07.116193 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:07.116020    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-8f1d78db-openshift-default-6bcc897b9-r5blj" podUID="3e4b6a7f-8bfd-49d8-be83-674d091c99d4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.113:15021/healthz/ready\": dial tcp 10.133.0.113:15021: connect: connection refused"
Oct 30 22:12:07.173355 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:07.173272    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-wrxqs"]
Oct 30 22:12:07.236418 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:07.217742    4473 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-wrxqs"]
Oct 30 22:12:07.292204 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:07.292174    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w6-5d36f6-openshift-default-555f9495c9-9lsxw"
Oct 30 22:12:07.292769 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:07.292743    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w6-5d36f6-openshift-default-555f9495c9-9lsxw"
Oct 30 22:12:07.293460 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:07.293427    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w6-5d36f6-openshift-default-555f9495c9-9lsxw"
Oct 30 22:12:07.355041 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:07.355005    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-j5tqr"]
Oct 30 22:12:07.358271 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:07.358248    4473 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-0f42cb-openshift-default-584f4c4865-j5tqr"]
Oct 30 22:12:08.121400 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:08.121366    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-8f1d78db-openshift-default-6bcc897b9-r5blj"
Oct 30 22:12:08.122716 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:08.122694    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-8f1d78db-openshift-default-6bcc897b9-r5blj"
Oct 30 22:12:08.359086 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:08.359053    4473 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0f0fbc43-d98b-4b7c-872b-7978f76650ef" path="/var/lib/kubelet/pods/0f0fbc43-d98b-4b7c-872b-7978f76650ef/volumes"
Oct 30 22:12:08.359766 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:08.359744    4473 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="370d0319-cf46-40d6-8c55-e4fa23e33dd3" path="/var/lib/kubelet/pods/370d0319-cf46-40d6-8c55-e4fa23e33dd3/volumes"
Oct 30 22:12:08.360288 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:08.360266    4473 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="78f06b05-32bf-46bc-bd95-2b3a8da72b5b" path="/var/lib/kubelet/pods/78f06b05-32bf-46bc-bd95-2b3a8da72b5b/volumes"
Oct 30 22:12:08.361107 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:08.361087    4473 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="85b6bb64-5545-4a61-bb0d-a4851348fe30" path="/var/lib/kubelet/pods/85b6bb64-5545-4a61-bb0d-a4851348fe30/volumes"
Oct 30 22:12:11.378431 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:11.378392    4473 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-ebddcc-867b88b67c-b66lr"]
Oct 30 22:12:11.379135 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:11.379119    4473 cpu_manager.go:414] "containerMap: removing container" podUID="85b6bb64-5545-4a61-bb0d-a4851348fe30" containerName="istio-proxy"
Oct 30 22:12:11.379193 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:11.379136    4473 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="85b6bb64-5545-4a61-bb0d-a4851348fe30" containerName="istio-proxy"
Oct 30 22:12:11.379193 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:11.379157    4473 cpu_manager.go:414] "containerMap: removing container" podUID="6560c1bc-b507-4a92-8586-23e0fcc5362a" containerName="istio-proxy"
Oct 30 22:12:11.379193 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:11.379162    4473 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6560c1bc-b507-4a92-8586-23e0fcc5362a" containerName="istio-proxy"
Oct 30 22:12:11.379193 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:11.379173    4473 cpu_manager.go:414] "containerMap: removing container" podUID="0f0fbc43-d98b-4b7c-872b-7978f76650ef" containerName="istio-proxy"
Oct 30 22:12:11.379193 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:11.379178    4473 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0f0fbc43-d98b-4b7c-872b-7978f76650ef" containerName="istio-proxy"
Oct 30 22:12:11.379193 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:11.379193    4473 cpu_manager.go:414] "containerMap: removing container" podUID="0f0fbc43-d98b-4b7c-872b-7978f76650ef" containerName="istio-proxy"
Oct 30 22:12:11.379377 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:11.379198    4473 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0f0fbc43-d98b-4b7c-872b-7978f76650ef" containerName="istio-proxy"
Oct 30 22:12:11.379377 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:11.379209    4473 cpu_manager.go:414] "containerMap: removing container" podUID="78f06b05-32bf-46bc-bd95-2b3a8da72b5b" containerName="istio-proxy"
Oct 30 22:12:11.379377 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:11.379214    4473 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="78f06b05-32bf-46bc-bd95-2b3a8da72b5b" containerName="istio-proxy"
Oct 30 22:12:11.379377 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:11.379222    4473 cpu_manager.go:414] "containerMap: removing container" podUID="370d0319-cf46-40d6-8c55-e4fa23e33dd3" containerName="istio-proxy"
Oct 30 22:12:11.379377 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:11.379226    4473 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="370d0319-cf46-40d6-8c55-e4fa23e33dd3" containerName="istio-proxy"
Oct 30 22:12:11.379377 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:11.379333    4473 memory_manager.go:365] "RemoveStaleState removing state" podUID="85b6bb64-5545-4a61-bb0d-a4851348fe30" containerName="istio-proxy"
Oct 30 22:12:11.379377 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:11.379344    4473 memory_manager.go:365] "RemoveStaleState removing state" podUID="6560c1bc-b507-4a92-8586-23e0fcc5362a" containerName="istio-proxy"
Oct 30 22:12:11.379377 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:11.379353    4473 memory_manager.go:365] "RemoveStaleState removing state" podUID="0f0fbc43-d98b-4b7c-872b-7978f76650ef" containerName="istio-proxy"
Oct 30 22:12:11.379377 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:11.379360    4473 memory_manager.go:365] "RemoveStaleState removing state" podUID="0f0fbc43-d98b-4b7c-872b-7978f76650ef" containerName="istio-proxy"
Oct 30 22:12:11.379377 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:11.379367    4473 memory_manager.go:365] "RemoveStaleState removing state" podUID="78f06b05-32bf-46bc-bd95-2b3a8da72b5b" containerName="istio-proxy"
Oct 30 22:12:11.379377 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:11.379373    4473 memory_manager.go:365] "RemoveStaleState removing state" podUID="370d0319-cf46-40d6-8c55-e4fa23e33dd3" containerName="istio-proxy"
Oct 30 22:12:11.383472 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:11.383451    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-ebddcc-867b88b67c-b66lr"
Oct 30 22:12:11.431867 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:11.431830    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-ebddcc-867b88b67c-b66lr"]
Oct 30 22:12:11.443026 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:11.442986    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w0-ebddcc-serving-cert\""
Oct 30 22:12:11.486891 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:11.486853    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pvcwh\" (UniqueName: \"kubernetes.io/projected/4c3f076c-c17f-4c7a-af38-10a974ced61d-kube-api-access-pvcwh\") pod \"maas-api-e2e-shared-a-w0-ebddcc-867b88b67c-b66lr\" (UID: \"4c3f076c-c17f-4c7a-af38-10a974ced61d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-ebddcc-867b88b67c-b66lr"
Oct 30 22:12:11.487487 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:11.487103    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4c3f076c-c17f-4c7a-af38-10a974ced61d-metrics-certs\") pod \"maas-api-e2e-shared-a-w0-ebddcc-867b88b67c-b66lr\" (UID: \"4c3f076c-c17f-4c7a-af38-10a974ced61d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-ebddcc-867b88b67c-b66lr"
Oct 30 22:12:11.487487 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:11.487253    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4c3f076c-c17f-4c7a-af38-10a974ced61d-maas-api-tls\") pod \"maas-api-e2e-shared-a-w0-ebddcc-867b88b67c-b66lr\" (UID: \"4c3f076c-c17f-4c7a-af38-10a974ced61d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-ebddcc-867b88b67c-b66lr"
Oct 30 22:12:11.487487 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:11.487295    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4c3f076c-c17f-4c7a-af38-10a974ced61d-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w0-ebddcc-867b88b67c-b66lr\" (UID: \"4c3f076c-c17f-4c7a-af38-10a974ced61d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-ebddcc-867b88b67c-b66lr"
Oct 30 22:12:11.588871 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:11.588833    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4c3f076c-c17f-4c7a-af38-10a974ced61d-metrics-certs\") pod \"maas-api-e2e-shared-a-w0-ebddcc-867b88b67c-b66lr\" (UID: \"4c3f076c-c17f-4c7a-af38-10a974ced61d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-ebddcc-867b88b67c-b66lr"
Oct 30 22:12:11.589116 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:11.588910    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4c3f076c-c17f-4c7a-af38-10a974ced61d-maas-api-tls\") pod \"maas-api-e2e-shared-a-w0-ebddcc-867b88b67c-b66lr\" (UID: \"4c3f076c-c17f-4c7a-af38-10a974ced61d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-ebddcc-867b88b67c-b66lr"
Oct 30 22:12:11.589116 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:11.588978    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4c3f076c-c17f-4c7a-af38-10a974ced61d-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w0-ebddcc-867b88b67c-b66lr\" (UID: \"4c3f076c-c17f-4c7a-af38-10a974ced61d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-ebddcc-867b88b67c-b66lr"
Oct 30 22:12:11.589116 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:11.589092    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pvcwh\" (UniqueName: \"kubernetes.io/projected/4c3f076c-c17f-4c7a-af38-10a974ced61d-kube-api-access-pvcwh\") pod \"maas-api-e2e-shared-a-w0-ebddcc-867b88b67c-b66lr\" (UID: \"4c3f076c-c17f-4c7a-af38-10a974ced61d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-ebddcc-867b88b67c-b66lr"
Oct 30 22:12:11.589686 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:11.589659    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4c3f076c-c17f-4c7a-af38-10a974ced61d-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w0-ebddcc-867b88b67c-b66lr\" (UID: \"4c3f076c-c17f-4c7a-af38-10a974ced61d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-ebddcc-867b88b67c-b66lr"
Oct 30 22:12:11.591584 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:11.591564    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4c3f076c-c17f-4c7a-af38-10a974ced61d-maas-api-tls\") pod \"maas-api-e2e-shared-a-w0-ebddcc-867b88b67c-b66lr\" (UID: \"4c3f076c-c17f-4c7a-af38-10a974ced61d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-ebddcc-867b88b67c-b66lr"
Oct 30 22:12:11.591651 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:11.591632    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4c3f076c-c17f-4c7a-af38-10a974ced61d-metrics-certs\") pod \"maas-api-e2e-shared-a-w0-ebddcc-867b88b67c-b66lr\" (UID: \"4c3f076c-c17f-4c7a-af38-10a974ced61d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-ebddcc-867b88b67c-b66lr"
Oct 30 22:12:11.652235 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:11.652158    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pvcwh\" (UniqueName: \"kubernetes.io/projected/4c3f076c-c17f-4c7a-af38-10a974ced61d-kube-api-access-pvcwh\") pod \"maas-api-e2e-shared-a-w0-ebddcc-867b88b67c-b66lr\" (UID: \"4c3f076c-c17f-4c7a-af38-10a974ced61d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-ebddcc-867b88b67c-b66lr"
Oct 30 22:12:11.697089 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:11.697052    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-ebddcc-867b88b67c-b66lr"
Oct 30 22:12:12.094183 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:12.094107    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-ebddcc-867b88b67c-b66lr"]
Oct 30 22:12:12.097579 ip-10-0-140-18 kubenswrapper[4473]: W1030 22:12:12.097549    4473 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c3f076c_c17f_4c7a_af38_10a974ced61d.slice/crio-dcb40eec6ba95e11954f88477fbcbdcbd936e849eb7e3a0b8912da4564d4b3e0 WatchSource:0}: Error finding container dcb40eec6ba95e11954f88477fbcbdcbd936e849eb7e3a0b8912da4564d4b3e0: Status 404 returned error can't find the container with id dcb40eec6ba95e11954f88477fbcbdcbd936e849eb7e3a0b8912da4564d4b3e0
Oct 30 22:12:12.497163 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:12.497125    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-ebddcc-867b88b67c-b66lr" event={"ID":"4c3f076c-c17f-4c7a-af38-10a974ced61d","Type":"ContainerStarted","Data":"dcb40eec6ba95e11954f88477fbcbdcbd936e849eb7e3a0b8912da4564d4b3e0"}
Oct 30 22:12:13.510620 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:13.510581    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-ebddcc-867b88b67c-b66lr" event={"ID":"4c3f076c-c17f-4c7a-af38-10a974ced61d","Type":"ContainerStarted","Data":"7397a43ed567843738c486f5e168ace0a2b51d64c1045e46486e1ba5430e2ed3"}
Oct 30 22:12:13.511125 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:13.510664    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-ebddcc-867b88b67c-b66lr"
Oct 30 22:12:13.683205 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:13.683153    4473 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-ebddcc-867b88b67c-b66lr" podStartSLOduration=1.9232875470000002 podStartE2EDuration="2.683137066s" totalImagesPullingTime="759.849519ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:12:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:12:13.679734997 +0000 UTC m=+1779.453103484" watchObservedRunningTime="2026-10-30 22:12:13.683137066 +0000 UTC m=+1779.456505552"
Oct 30 22:12:14.895625 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:14.895589    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-242ddaf0/e2e-lifecycle-model-242ddaf0-kserve-87686648f-l9tjf"
Oct 30 22:12:14.908892 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:14.908859    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-242ddaf0/e2e-lifecycle-model-242ddaf0-kserve-87686648f-l9tjf"
Oct 30 22:12:15.547333 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:15.547296    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-242ddaf0/e2e-lifecycle-model-242ddaf0-kserve-87686648f-l9tjf"
Oct 30 22:12:19.523859 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:19.523829    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-ebddcc-867b88b67c-b66lr"
Oct 30 22:12:20.325816 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:12:20.325782    4473 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fncvg" containerName="extractor"
Oct 30 22:12:22.469380 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:22.469346    4473 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-7c826c-f77fdff44-mtg87"]
Oct 30 22:12:22.474326 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:22.474305    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-7c826c-f77fdff44-mtg87"
Oct 30 22:12:22.530508 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:22.530472    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-7c826c-f77fdff44-mtg87"]
Oct 30 22:12:22.598334 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:22.598303    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w2-7c826c-serving-cert\""
Oct 30 22:12:22.639338 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:22.639301    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/021c5c2e-7d54-46be-8639-9183b8021972-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-7c826c-f77fdff44-mtg87\" (UID: \"021c5c2e-7d54-46be-8639-9183b8021972\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-7c826c-f77fdff44-mtg87"
Oct 30 22:12:22.639542 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:22.639473    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/021c5c2e-7d54-46be-8639-9183b8021972-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-7c826c-f77fdff44-mtg87\" (UID: \"021c5c2e-7d54-46be-8639-9183b8021972\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-7c826c-f77fdff44-mtg87"
Oct 30 22:12:22.639615 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:22.639592    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/021c5c2e-7d54-46be-8639-9183b8021972-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-7c826c-f77fdff44-mtg87\" (UID: \"021c5c2e-7d54-46be-8639-9183b8021972\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-7c826c-f77fdff44-mtg87"
Oct 30 22:12:22.639715 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:22.639693    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hmqqb\" (UniqueName: \"kubernetes.io/projected/021c5c2e-7d54-46be-8639-9183b8021972-kube-api-access-hmqqb\") pod \"maas-api-e2e-shared-b-w2-7c826c-f77fdff44-mtg87\" (UID: \"021c5c2e-7d54-46be-8639-9183b8021972\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-7c826c-f77fdff44-mtg87"
Oct 30 22:12:22.752300 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:22.752215    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/021c5c2e-7d54-46be-8639-9183b8021972-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-7c826c-f77fdff44-mtg87\" (UID: \"021c5c2e-7d54-46be-8639-9183b8021972\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-7c826c-f77fdff44-mtg87"
Oct 30 22:12:22.772656 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:22.772616    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/021c5c2e-7d54-46be-8639-9183b8021972-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-7c826c-f77fdff44-mtg87\" (UID: \"021c5c2e-7d54-46be-8639-9183b8021972\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-7c826c-f77fdff44-mtg87"
Oct 30 22:12:22.775391 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:22.775361    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/021c5c2e-7d54-46be-8639-9183b8021972-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-7c826c-f77fdff44-mtg87\" (UID: \"021c5c2e-7d54-46be-8639-9183b8021972\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-7c826c-f77fdff44-mtg87"
Oct 30 22:12:22.776022 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:22.774773    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/021c5c2e-7d54-46be-8639-9183b8021972-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-7c826c-f77fdff44-mtg87\" (UID: \"021c5c2e-7d54-46be-8639-9183b8021972\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-7c826c-f77fdff44-mtg87"
Oct 30 22:12:22.776239 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:22.776216    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hmqqb\" (UniqueName: \"kubernetes.io/projected/021c5c2e-7d54-46be-8639-9183b8021972-kube-api-access-hmqqb\") pod \"maas-api-e2e-shared-b-w2-7c826c-f77fdff44-mtg87\" (UID: \"021c5c2e-7d54-46be-8639-9183b8021972\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-7c826c-f77fdff44-mtg87"
Oct 30 22:12:22.795766 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:22.795732    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/021c5c2e-7d54-46be-8639-9183b8021972-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-7c826c-f77fdff44-mtg87\" (UID: \"021c5c2e-7d54-46be-8639-9183b8021972\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-7c826c-f77fdff44-mtg87"
Oct 30 22:12:22.799385 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:22.799321    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/021c5c2e-7d54-46be-8639-9183b8021972-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-7c826c-f77fdff44-mtg87\" (UID: \"021c5c2e-7d54-46be-8639-9183b8021972\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-7c826c-f77fdff44-mtg87"
Oct 30 22:12:22.837381 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:22.837351    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hmqqb\" (UniqueName: \"kubernetes.io/projected/021c5c2e-7d54-46be-8639-9183b8021972-kube-api-access-hmqqb\") pod \"maas-api-e2e-shared-b-w2-7c826c-f77fdff44-mtg87\" (UID: \"021c5c2e-7d54-46be-8639-9183b8021972\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-7c826c-f77fdff44-mtg87"
Oct 30 22:12:23.088874 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:23.088796    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-7c826c-f77fdff44-mtg87"
Oct 30 22:12:23.477370 ip-10-0-140-18 kubenswrapper[4473]: W1030 22:12:23.477343    4473 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod021c5c2e_7d54_46be_8639_9183b8021972.slice/crio-ab34bf4deba863523ff60a176391f997ac2246c32bdcc97218444db1a1f37259 WatchSource:0}: Error finding container ab34bf4deba863523ff60a176391f997ac2246c32bdcc97218444db1a1f37259: Status 404 returned error can't find the container with id ab34bf4deba863523ff60a176391f997ac2246c32bdcc97218444db1a1f37259
Oct 30 22:12:23.479194 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:23.479171    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-7c826c-f77fdff44-mtg87"]
Oct 30 22:12:23.647988 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:23.647950    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-7c826c-f77fdff44-mtg87" event={"ID":"021c5c2e-7d54-46be-8639-9183b8021972","Type":"ContainerStarted","Data":"ab34bf4deba863523ff60a176391f997ac2246c32bdcc97218444db1a1f37259"}
Oct 30 22:12:24.657313 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:24.657278    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-7c826c-f77fdff44-mtg87" event={"ID":"021c5c2e-7d54-46be-8639-9183b8021972","Type":"ContainerStarted","Data":"258d12f772b06455812f87f9d3a19c43732596aafc8ba3ce377b22e7dfa8b1ed"}
Oct 30 22:12:24.657715 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:24.657488    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-7c826c-f77fdff44-mtg87"
Oct 30 22:12:24.794146 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:24.794081    4473 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-7c826c-f77fdff44-mtg87" podStartSLOduration=1.970576025 podStartE2EDuration="2.794067955s" totalImagesPullingTime="823.49193ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:12:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:12:24.789866411 +0000 UTC m=+1790.563234898" watchObservedRunningTime="2026-10-30 22:12:24.794067955 +0000 UTC m=+1790.567436442"
Oct 30 22:12:27.670614 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:27.670572    4473 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-e431fc-5bb8bbd5f9-dgv2g"]
Oct 30 22:12:27.675119 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:27.675093    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-e431fc-5bb8bbd5f9-dgv2g"
Oct 30 22:12:27.760090 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:27.760051    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n2clb\" (UniqueName: \"kubernetes.io/projected/806e3639-73c0-4dc2-87bf-4a4a95444e49-kube-api-access-n2clb\") pod \"maas-api-e2e-shared-a-w5-e431fc-5bb8bbd5f9-dgv2g\" (UID: \"806e3639-73c0-4dc2-87bf-4a4a95444e49\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-e431fc-5bb8bbd5f9-dgv2g"
Oct 30 22:12:27.760318 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:27.760245    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/806e3639-73c0-4dc2-87bf-4a4a95444e49-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w5-e431fc-5bb8bbd5f9-dgv2g\" (UID: \"806e3639-73c0-4dc2-87bf-4a4a95444e49\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-e431fc-5bb8bbd5f9-dgv2g"
Oct 30 22:12:27.760318 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:27.760296    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/806e3639-73c0-4dc2-87bf-4a4a95444e49-maas-api-tls\") pod \"maas-api-e2e-shared-a-w5-e431fc-5bb8bbd5f9-dgv2g\" (UID: \"806e3639-73c0-4dc2-87bf-4a4a95444e49\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-e431fc-5bb8bbd5f9-dgv2g"
Oct 30 22:12:27.760487 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:27.760330    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/806e3639-73c0-4dc2-87bf-4a4a95444e49-metrics-certs\") pod \"maas-api-e2e-shared-a-w5-e431fc-5bb8bbd5f9-dgv2g\" (UID: \"806e3639-73c0-4dc2-87bf-4a4a95444e49\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-e431fc-5bb8bbd5f9-dgv2g"
Oct 30 22:12:27.778717 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:27.778674    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w5-e431fc-serving-cert\""
Oct 30 22:12:27.782806 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:27.782775    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-e431fc-5bb8bbd5f9-dgv2g"]
Oct 30 22:12:27.861359 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:27.861326    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/806e3639-73c0-4dc2-87bf-4a4a95444e49-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w5-e431fc-5bb8bbd5f9-dgv2g\" (UID: \"806e3639-73c0-4dc2-87bf-4a4a95444e49\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-e431fc-5bb8bbd5f9-dgv2g"
Oct 30 22:12:27.861563 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:27.861379    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/806e3639-73c0-4dc2-87bf-4a4a95444e49-maas-api-tls\") pod \"maas-api-e2e-shared-a-w5-e431fc-5bb8bbd5f9-dgv2g\" (UID: \"806e3639-73c0-4dc2-87bf-4a4a95444e49\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-e431fc-5bb8bbd5f9-dgv2g"
Oct 30 22:12:27.861563 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:27.861405    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/806e3639-73c0-4dc2-87bf-4a4a95444e49-metrics-certs\") pod \"maas-api-e2e-shared-a-w5-e431fc-5bb8bbd5f9-dgv2g\" (UID: \"806e3639-73c0-4dc2-87bf-4a4a95444e49\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-e431fc-5bb8bbd5f9-dgv2g"
Oct 30 22:12:27.861563 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:27.861491    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n2clb\" (UniqueName: \"kubernetes.io/projected/806e3639-73c0-4dc2-87bf-4a4a95444e49-kube-api-access-n2clb\") pod \"maas-api-e2e-shared-a-w5-e431fc-5bb8bbd5f9-dgv2g\" (UID: \"806e3639-73c0-4dc2-87bf-4a4a95444e49\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-e431fc-5bb8bbd5f9-dgv2g"
Oct 30 22:12:27.861563 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:12:27.861545    4473 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-e431fc-serving-cert: secret "maas-api-e2e-shared-a-w5-e431fc-serving-cert" not found
Oct 30 22:12:27.861783 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:12:27.861636    4473 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/806e3639-73c0-4dc2-87bf-4a4a95444e49-maas-api-tls podName:806e3639-73c0-4dc2-87bf-4a4a95444e49 nodeName:}" failed. No retries permitted until 2026-10-30 22:12:28.361610558 +0000 UTC m=+1794.134979040 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/806e3639-73c0-4dc2-87bf-4a4a95444e49-maas-api-tls") pod "maas-api-e2e-shared-a-w5-e431fc-5bb8bbd5f9-dgv2g" (UID: "806e3639-73c0-4dc2-87bf-4a4a95444e49") : secret "maas-api-e2e-shared-a-w5-e431fc-serving-cert" not found
Oct 30 22:12:27.862113 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:27.862095    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/806e3639-73c0-4dc2-87bf-4a4a95444e49-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w5-e431fc-5bb8bbd5f9-dgv2g\" (UID: \"806e3639-73c0-4dc2-87bf-4a4a95444e49\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-e431fc-5bb8bbd5f9-dgv2g"
Oct 30 22:12:27.864256 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:27.864239    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/806e3639-73c0-4dc2-87bf-4a4a95444e49-metrics-certs\") pod \"maas-api-e2e-shared-a-w5-e431fc-5bb8bbd5f9-dgv2g\" (UID: \"806e3639-73c0-4dc2-87bf-4a4a95444e49\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-e431fc-5bb8bbd5f9-dgv2g"
Oct 30 22:12:27.999993 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:27.999080    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n2clb\" (UniqueName: \"kubernetes.io/projected/806e3639-73c0-4dc2-87bf-4a4a95444e49-kube-api-access-n2clb\") pod \"maas-api-e2e-shared-a-w5-e431fc-5bb8bbd5f9-dgv2g\" (UID: \"806e3639-73c0-4dc2-87bf-4a4a95444e49\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-e431fc-5bb8bbd5f9-dgv2g"
Oct 30 22:12:28.371205 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:28.371170    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/806e3639-73c0-4dc2-87bf-4a4a95444e49-maas-api-tls\") pod \"maas-api-e2e-shared-a-w5-e431fc-5bb8bbd5f9-dgv2g\" (UID: \"806e3639-73c0-4dc2-87bf-4a4a95444e49\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-e431fc-5bb8bbd5f9-dgv2g"
Oct 30 22:12:28.373896 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:28.373865    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/806e3639-73c0-4dc2-87bf-4a4a95444e49-maas-api-tls\") pod \"maas-api-e2e-shared-a-w5-e431fc-5bb8bbd5f9-dgv2g\" (UID: \"806e3639-73c0-4dc2-87bf-4a4a95444e49\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-e431fc-5bb8bbd5f9-dgv2g"
Oct 30 22:12:28.590617 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:28.590581    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-e431fc-5bb8bbd5f9-dgv2g"
Oct 30 22:12:28.975818 ip-10-0-140-18 kubenswrapper[4473]: W1030 22:12:28.975788    4473 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod806e3639_73c0_4dc2_87bf_4a4a95444e49.slice/crio-0c6c26be27447a74301660dac06c853921d7391eda613cfdd32fe23335589485 WatchSource:0}: Error finding container 0c6c26be27447a74301660dac06c853921d7391eda613cfdd32fe23335589485: Status 404 returned error can't find the container with id 0c6c26be27447a74301660dac06c853921d7391eda613cfdd32fe23335589485
Oct 30 22:12:28.976279 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:28.975950    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-e431fc-5bb8bbd5f9-dgv2g"]
Oct 30 22:12:29.698579 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:29.698542    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-e431fc-5bb8bbd5f9-dgv2g" event={"ID":"806e3639-73c0-4dc2-87bf-4a4a95444e49","Type":"ContainerStarted","Data":"0c6c26be27447a74301660dac06c853921d7391eda613cfdd32fe23335589485"}
Oct 30 22:12:30.671321 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:30.671291    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-7c826c-f77fdff44-mtg87"
Oct 30 22:12:30.707644 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:30.707611    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-e431fc-5bb8bbd5f9-dgv2g" event={"ID":"806e3639-73c0-4dc2-87bf-4a4a95444e49","Type":"ContainerStarted","Data":"1c86b6a4c299895072f9e0fa08affdb99c0553a35b96a9364a9b4f069eaee262"}
Oct 30 22:12:30.707820 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:30.707678    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-e431fc-5bb8bbd5f9-dgv2g"
Oct 30 22:12:30.932804 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:30.930644    4473 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-e431fc-5bb8bbd5f9-dgv2g" podStartSLOduration=3.13930512 podStartE2EDuration="3.930623872s" totalImagesPullingTime="791.318752ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:12:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:12:30.929371361 +0000 UTC m=+1796.702739849" watchObservedRunningTime="2026-10-30 22:12:30.930623872 +0000 UTC m=+1796.703992360"
Oct 30 22:12:36.720701 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:36.720666    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-e431fc-5bb8bbd5f9-dgv2g"
Oct 30 22:12:41.242087 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:41.242040    4473 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w6-0a7679-openshift-default-5b6c6cc645-p8j4w"]
Oct 30 22:12:41.533369 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:41.533282    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w6-0a7679-openshift-default-5b6c6cc645-p8j4w"
Oct 30 22:12:41.546946 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:41.538481    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w6-0a7679-openshift-default-5b6c6cc645-p8j4w"]
Oct 30 22:12:41.591937 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:41.591886    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b7789a8c-e252-4a6e-b026-3b5ce2b39b41-istio-token\") pod \"e2e-shared-b-w6-0a7679-openshift-default-5b6c6cc645-p8j4w\" (UID: \"b7789a8c-e252-4a6e-b026-3b5ce2b39b41\") " pod="openshift-ingress/e2e-shared-b-w6-0a7679-openshift-default-5b6c6cc645-p8j4w"
Oct 30 22:12:41.592130 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:41.591997    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-26n5z\" (UniqueName: \"kubernetes.io/projected/b7789a8c-e252-4a6e-b026-3b5ce2b39b41-kube-api-access-26n5z\") pod \"e2e-shared-b-w6-0a7679-openshift-default-5b6c6cc645-p8j4w\" (UID: \"b7789a8c-e252-4a6e-b026-3b5ce2b39b41\") " pod="openshift-ingress/e2e-shared-b-w6-0a7679-openshift-default-5b6c6cc645-p8j4w"
Oct 30 22:12:41.592130 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:41.592039    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b7789a8c-e252-4a6e-b026-3b5ce2b39b41-workload-certs\") pod \"e2e-shared-b-w6-0a7679-openshift-default-5b6c6cc645-p8j4w\" (UID: \"b7789a8c-e252-4a6e-b026-3b5ce2b39b41\") " pod="openshift-ingress/e2e-shared-b-w6-0a7679-openshift-default-5b6c6cc645-p8j4w"
Oct 30 22:12:41.592130 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:41.592058    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b7789a8c-e252-4a6e-b026-3b5ce2b39b41-istio-podinfo\") pod \"e2e-shared-b-w6-0a7679-openshift-default-5b6c6cc645-p8j4w\" (UID: \"b7789a8c-e252-4a6e-b026-3b5ce2b39b41\") " pod="openshift-ingress/e2e-shared-b-w6-0a7679-openshift-default-5b6c6cc645-p8j4w"
Oct 30 22:12:41.592306 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:41.592284    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7789a8c-e252-4a6e-b026-3b5ce2b39b41-credential-socket\") pod \"e2e-shared-b-w6-0a7679-openshift-default-5b6c6cc645-p8j4w\" (UID: \"b7789a8c-e252-4a6e-b026-3b5ce2b39b41\") " pod="openshift-ingress/e2e-shared-b-w6-0a7679-openshift-default-5b6c6cc645-p8j4w"
Oct 30 22:12:41.592471 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:41.592452    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7789a8c-e252-4a6e-b026-3b5ce2b39b41-workload-socket\") pod \"e2e-shared-b-w6-0a7679-openshift-default-5b6c6cc645-p8j4w\" (UID: \"b7789a8c-e252-4a6e-b026-3b5ce2b39b41\") " pod="openshift-ingress/e2e-shared-b-w6-0a7679-openshift-default-5b6c6cc645-p8j4w"
Oct 30 22:12:41.592545 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:41.592503    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b7789a8c-e252-4a6e-b026-3b5ce2b39b41-istio-envoy\") pod \"e2e-shared-b-w6-0a7679-openshift-default-5b6c6cc645-p8j4w\" (UID: \"b7789a8c-e252-4a6e-b026-3b5ce2b39b41\") " pod="openshift-ingress/e2e-shared-b-w6-0a7679-openshift-default-5b6c6cc645-p8j4w"
Oct 30 22:12:41.592545 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:41.592530    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b7789a8c-e252-4a6e-b026-3b5ce2b39b41-istio-data\") pod \"e2e-shared-b-w6-0a7679-openshift-default-5b6c6cc645-p8j4w\" (UID: \"b7789a8c-e252-4a6e-b026-3b5ce2b39b41\") " pod="openshift-ingress/e2e-shared-b-w6-0a7679-openshift-default-5b6c6cc645-p8j4w"
Oct 30 22:12:41.592744 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:41.592722    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b7789a8c-e252-4a6e-b026-3b5ce2b39b41-istiod-ca-cert\") pod \"e2e-shared-b-w6-0a7679-openshift-default-5b6c6cc645-p8j4w\" (UID: \"b7789a8c-e252-4a6e-b026-3b5ce2b39b41\") " pod="openshift-ingress/e2e-shared-b-w6-0a7679-openshift-default-5b6c6cc645-p8j4w"
Oct 30 22:12:41.641763 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:41.641714    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w6-0a7679-openshift-default-dockercfg-nfdql\""
Oct 30 22:12:41.699130 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:41.699086    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b7789a8c-e252-4a6e-b026-3b5ce2b39b41-istio-token\") pod \"e2e-shared-b-w6-0a7679-openshift-default-5b6c6cc645-p8j4w\" (UID: \"b7789a8c-e252-4a6e-b026-3b5ce2b39b41\") " pod="openshift-ingress/e2e-shared-b-w6-0a7679-openshift-default-5b6c6cc645-p8j4w"
Oct 30 22:12:41.703075 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:41.703023    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-26n5z\" (UniqueName: \"kubernetes.io/projected/b7789a8c-e252-4a6e-b026-3b5ce2b39b41-kube-api-access-26n5z\") pod \"e2e-shared-b-w6-0a7679-openshift-default-5b6c6cc645-p8j4w\" (UID: \"b7789a8c-e252-4a6e-b026-3b5ce2b39b41\") " pod="openshift-ingress/e2e-shared-b-w6-0a7679-openshift-default-5b6c6cc645-p8j4w"
Oct 30 22:12:41.703351 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:41.703333    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b7789a8c-e252-4a6e-b026-3b5ce2b39b41-workload-certs\") pod \"e2e-shared-b-w6-0a7679-openshift-default-5b6c6cc645-p8j4w\" (UID: \"b7789a8c-e252-4a6e-b026-3b5ce2b39b41\") " pod="openshift-ingress/e2e-shared-b-w6-0a7679-openshift-default-5b6c6cc645-p8j4w"
Oct 30 22:12:41.703462 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:41.703449    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b7789a8c-e252-4a6e-b026-3b5ce2b39b41-istio-podinfo\") pod \"e2e-shared-b-w6-0a7679-openshift-default-5b6c6cc645-p8j4w\" (UID: \"b7789a8c-e252-4a6e-b026-3b5ce2b39b41\") " pod="openshift-ingress/e2e-shared-b-w6-0a7679-openshift-default-5b6c6cc645-p8j4w"
Oct 30 22:12:41.703737 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:41.703713    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7789a8c-e252-4a6e-b026-3b5ce2b39b41-credential-socket\") pod \"e2e-shared-b-w6-0a7679-openshift-default-5b6c6cc645-p8j4w\" (UID: \"b7789a8c-e252-4a6e-b026-3b5ce2b39b41\") " pod="openshift-ingress/e2e-shared-b-w6-0a7679-openshift-default-5b6c6cc645-p8j4w"
Oct 30 22:12:41.712452 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:41.703982    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7789a8c-e252-4a6e-b026-3b5ce2b39b41-workload-socket\") pod \"e2e-shared-b-w6-0a7679-openshift-default-5b6c6cc645-p8j4w\" (UID: \"b7789a8c-e252-4a6e-b026-3b5ce2b39b41\") " pod="openshift-ingress/e2e-shared-b-w6-0a7679-openshift-default-5b6c6cc645-p8j4w"
Oct 30 22:12:41.712452 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:41.712142    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b7789a8c-e252-4a6e-b026-3b5ce2b39b41-istio-envoy\") pod \"e2e-shared-b-w6-0a7679-openshift-default-5b6c6cc645-p8j4w\" (UID: \"b7789a8c-e252-4a6e-b026-3b5ce2b39b41\") " pod="openshift-ingress/e2e-shared-b-w6-0a7679-openshift-default-5b6c6cc645-p8j4w"
Oct 30 22:12:41.712452 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:41.712182    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b7789a8c-e252-4a6e-b026-3b5ce2b39b41-istio-data\") pod \"e2e-shared-b-w6-0a7679-openshift-default-5b6c6cc645-p8j4w\" (UID: \"b7789a8c-e252-4a6e-b026-3b5ce2b39b41\") " pod="openshift-ingress/e2e-shared-b-w6-0a7679-openshift-default-5b6c6cc645-p8j4w"
Oct 30 22:12:41.712452 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:41.712332    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b7789a8c-e252-4a6e-b026-3b5ce2b39b41-istiod-ca-cert\") pod \"e2e-shared-b-w6-0a7679-openshift-default-5b6c6cc645-p8j4w\" (UID: \"b7789a8c-e252-4a6e-b026-3b5ce2b39b41\") " pod="openshift-ingress/e2e-shared-b-w6-0a7679-openshift-default-5b6c6cc645-p8j4w"
Oct 30 22:12:41.713283 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:41.713253    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b7789a8c-e252-4a6e-b026-3b5ce2b39b41-istiod-ca-cert\") pod \"e2e-shared-b-w6-0a7679-openshift-default-5b6c6cc645-p8j4w\" (UID: \"b7789a8c-e252-4a6e-b026-3b5ce2b39b41\") " pod="openshift-ingress/e2e-shared-b-w6-0a7679-openshift-default-5b6c6cc645-p8j4w"
Oct 30 22:12:41.713440 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:41.711944    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b7789a8c-e252-4a6e-b026-3b5ce2b39b41-workload-certs\") pod \"e2e-shared-b-w6-0a7679-openshift-default-5b6c6cc645-p8j4w\" (UID: \"b7789a8c-e252-4a6e-b026-3b5ce2b39b41\") " pod="openshift-ingress/e2e-shared-b-w6-0a7679-openshift-default-5b6c6cc645-p8j4w"
Oct 30 22:12:41.713598 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:41.713539    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b7789a8c-e252-4a6e-b026-3b5ce2b39b41-istio-data\") pod \"e2e-shared-b-w6-0a7679-openshift-default-5b6c6cc645-p8j4w\" (UID: \"b7789a8c-e252-4a6e-b026-3b5ce2b39b41\") " pod="openshift-ingress/e2e-shared-b-w6-0a7679-openshift-default-5b6c6cc645-p8j4w"
Oct 30 22:12:41.717211 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:41.713786    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7789a8c-e252-4a6e-b026-3b5ce2b39b41-credential-socket\") pod \"e2e-shared-b-w6-0a7679-openshift-default-5b6c6cc645-p8j4w\" (UID: \"b7789a8c-e252-4a6e-b026-3b5ce2b39b41\") " pod="openshift-ingress/e2e-shared-b-w6-0a7679-openshift-default-5b6c6cc645-p8j4w"
Oct 30 22:12:41.717403 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:41.711119    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7789a8c-e252-4a6e-b026-3b5ce2b39b41-workload-socket\") pod \"e2e-shared-b-w6-0a7679-openshift-default-5b6c6cc645-p8j4w\" (UID: \"b7789a8c-e252-4a6e-b026-3b5ce2b39b41\") " pod="openshift-ingress/e2e-shared-b-w6-0a7679-openshift-default-5b6c6cc645-p8j4w"
Oct 30 22:12:41.717403 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:41.716616    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b7789a8c-e252-4a6e-b026-3b5ce2b39b41-istio-podinfo\") pod \"e2e-shared-b-w6-0a7679-openshift-default-5b6c6cc645-p8j4w\" (UID: \"b7789a8c-e252-4a6e-b026-3b5ce2b39b41\") " pod="openshift-ingress/e2e-shared-b-w6-0a7679-openshift-default-5b6c6cc645-p8j4w"
Oct 30 22:12:41.720304 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:41.720181    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b7789a8c-e252-4a6e-b026-3b5ce2b39b41-istio-envoy\") pod \"e2e-shared-b-w6-0a7679-openshift-default-5b6c6cc645-p8j4w\" (UID: \"b7789a8c-e252-4a6e-b026-3b5ce2b39b41\") " pod="openshift-ingress/e2e-shared-b-w6-0a7679-openshift-default-5b6c6cc645-p8j4w"
Oct 30 22:12:41.770089 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:41.770054    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b7789a8c-e252-4a6e-b026-3b5ce2b39b41-istio-token\") pod \"e2e-shared-b-w6-0a7679-openshift-default-5b6c6cc645-p8j4w\" (UID: \"b7789a8c-e252-4a6e-b026-3b5ce2b39b41\") " pod="openshift-ingress/e2e-shared-b-w6-0a7679-openshift-default-5b6c6cc645-p8j4w"
Oct 30 22:12:41.770329 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:41.770305    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-26n5z\" (UniqueName: \"kubernetes.io/projected/b7789a8c-e252-4a6e-b026-3b5ce2b39b41-kube-api-access-26n5z\") pod \"e2e-shared-b-w6-0a7679-openshift-default-5b6c6cc645-p8j4w\" (UID: \"b7789a8c-e252-4a6e-b026-3b5ce2b39b41\") " pod="openshift-ingress/e2e-shared-b-w6-0a7679-openshift-default-5b6c6cc645-p8j4w"
Oct 30 22:12:41.861323 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:41.861249    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w6-0a7679-openshift-default-5b6c6cc645-p8j4w"
Oct 30 22:12:42.295241 ip-10-0-140-18 kubenswrapper[4473]: W1030 22:12:42.295209    4473 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7789a8c_e252_4a6e_b026_3b5ce2b39b41.slice/crio-9ecde4e0a5e17ad41702e166b8f327db178db8054b3b9c74d12027ef3461123d WatchSource:0}: Error finding container 9ecde4e0a5e17ad41702e166b8f327db178db8054b3b9c74d12027ef3461123d: Status 404 returned error can't find the container with id 9ecde4e0a5e17ad41702e166b8f327db178db8054b3b9c74d12027ef3461123d
Oct 30 22:12:42.295576 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:42.295297    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w6-0a7679-openshift-default-5b6c6cc645-p8j4w"]
Oct 30 22:12:42.298136 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:42.298099    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:12:42.298245 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:42.298166    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:12:42.298245 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:42.298195    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:12:42.799226 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:42.799187    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w6-0a7679-openshift-default-5b6c6cc645-p8j4w" event={"ID":"b7789a8c-e252-4a6e-b026-3b5ce2b39b41","Type":"ContainerStarted","Data":"29565cd3fb267390ae283bf15cf0534a08cb0883922fd44fc5f909778f42b88b"}
Oct 30 22:12:42.799412 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:42.799245    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w6-0a7679-openshift-default-5b6c6cc645-p8j4w" event={"ID":"b7789a8c-e252-4a6e-b026-3b5ce2b39b41","Type":"ContainerStarted","Data":"9ecde4e0a5e17ad41702e166b8f327db178db8054b3b9c74d12027ef3461123d"}
Oct 30 22:12:42.861862 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:42.861829    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w6-0a7679-openshift-default-5b6c6cc645-p8j4w"
Oct 30 22:12:42.921782 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:42.921728    4473 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w6-0a7679-openshift-default-5b6c6cc645-p8j4w" podStartSLOduration=1.921714667 podStartE2EDuration="1.921714667s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:12:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:12:42.919090713 +0000 UTC m=+1808.692459200" watchObservedRunningTime="2026-10-30 22:12:42.921714667 +0000 UTC m=+1808.695083154"
Oct 30 22:12:43.868853 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:43.868820    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w6-0a7679-openshift-default-5b6c6cc645-p8j4w"
Oct 30 22:12:44.814008 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:44.813978    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w6-0a7679-openshift-default-5b6c6cc645-p8j4w"
Oct 30 22:12:44.824400 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:44.824360    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w6-0a7679-openshift-default-5b6c6cc645-p8j4w"
Oct 30 22:12:49.838976 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:49.830880    4473 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w6-0a7679-69cc484995-hlw4x"]
Oct 30 22:12:49.838976 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:49.837836    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w6-0a7679-69cc484995-hlw4x"
Oct 30 22:12:49.872350 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:49.871840    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9e922331-4f21-49d7-88d3-7a48d2854cea-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w6-0a7679-69cc484995-hlw4x\" (UID: \"9e922331-4f21-49d7-88d3-7a48d2854cea\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w6-0a7679-69cc484995-hlw4x"
Oct 30 22:12:49.872350 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:49.871890    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d7wxt\" (UniqueName: \"kubernetes.io/projected/9e922331-4f21-49d7-88d3-7a48d2854cea-kube-api-access-d7wxt\") pod \"payload-pre-processing-e2e-shared-b-w6-0a7679-69cc484995-hlw4x\" (UID: \"9e922331-4f21-49d7-88d3-7a48d2854cea\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w6-0a7679-69cc484995-hlw4x"
Oct 30 22:12:49.929027 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:49.926695    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w6-0a7679-69cc484995-hlw4x"]
Oct 30 22:12:49.951938 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:49.947313    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w6-0a7679-dockercfg-cs4br\""
Oct 30 22:12:49.951938 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:49.947397    4473 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w6-0a7679\""
Oct 30 22:12:49.981965 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:49.980290    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9e922331-4f21-49d7-88d3-7a48d2854cea-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w6-0a7679-69cc484995-hlw4x\" (UID: \"9e922331-4f21-49d7-88d3-7a48d2854cea\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w6-0a7679-69cc484995-hlw4x"
Oct 30 22:12:49.981965 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:49.980350    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d7wxt\" (UniqueName: \"kubernetes.io/projected/9e922331-4f21-49d7-88d3-7a48d2854cea-kube-api-access-d7wxt\") pod \"payload-pre-processing-e2e-shared-b-w6-0a7679-69cc484995-hlw4x\" (UID: \"9e922331-4f21-49d7-88d3-7a48d2854cea\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w6-0a7679-69cc484995-hlw4x"
Oct 30 22:12:49.991941 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:49.988300    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9e922331-4f21-49d7-88d3-7a48d2854cea-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w6-0a7679-69cc484995-hlw4x\" (UID: \"9e922331-4f21-49d7-88d3-7a48d2854cea\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w6-0a7679-69cc484995-hlw4x"
Oct 30 22:12:50.161234 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:50.161196    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d7wxt\" (UniqueName: \"kubernetes.io/projected/9e922331-4f21-49d7-88d3-7a48d2854cea-kube-api-access-d7wxt\") pod \"payload-pre-processing-e2e-shared-b-w6-0a7679-69cc484995-hlw4x\" (UID: \"9e922331-4f21-49d7-88d3-7a48d2854cea\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w6-0a7679-69cc484995-hlw4x"
Oct 30 22:12:50.224103 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:50.222571    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w6-0a7679-69cc484995-hlw4x"
Oct 30 22:12:50.641627 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:12:50.632376    4473 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fncvg" containerName="extractor"
Oct 30 22:12:51.274945 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:51.265786    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w6-0a7679-69cc484995-hlw4x"]
Oct 30 22:12:52.233954 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:52.220989    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w6-0a7679-69cc484995-hlw4x" event={"ID":"9e922331-4f21-49d7-88d3-7a48d2854cea","Type":"ContainerStarted","Data":"73b26cc4e4691e234b566f9b4a1c515a22c1ef52aba646cfbeb66c788459b03c"}
Oct 30 22:12:52.233954 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:52.221057    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w6-0a7679-69cc484995-hlw4x" event={"ID":"9e922331-4f21-49d7-88d3-7a48d2854cea","Type":"ContainerStarted","Data":"a8cd11d0384d99aaa3d06cd1707ffbe46ad65f570a94c9601b80563662f9f328"}
Oct 30 22:12:52.236818 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:52.235169    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w6-0a7679-69cc484995-hlw4x"
Oct 30 22:12:53.051952 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:53.048088    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-dde1e8/distinct-2-w0-dde1e8-kserve-744857d57-ftt8b" podUID="577b020d-2a6b-4020-863c-cf19d4f0ca3b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.79:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:12:53.156973 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:53.149041    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-0c99b1/distinct-2-w2-0c99b1-kserve-7466864658-8z7fk" podUID="89c09555-cf53-49a7-9c3b-95d2da2a293a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.82:8000/ready\": context deadline exceeded"
Oct 30 22:12:53.182953 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:53.182474    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-0f42cb/distinct-w3-0f42cb-kserve-fb54fdd59-n6w4l" podUID="ff3c2b25-08aa-4b70-bf57-9cf4a11d05ed" containerName="main" probeResult="failure" output="Get \"https://10.133.0.73:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:12:53.255946 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:53.252708    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-9f8aea/distinct-w5-9f8aea-kserve-5b564579cd-kmxzz" podUID="b11e8b11-334f-4a9e-98c2-b0164567a8af" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:12:53.324007 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:53.315544    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-0f42cb/distinct-2-w3-0f42cb-kserve-fc477897b-jk8qc" podUID="61b49182-057e-41af-b609-1a6abf82c197" containerName="main" probeResult="failure" output="Get \"https://10.133.0.80:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:12:53.530948 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:53.530028    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-mt-242ddaf0/e2e-lifecycle-model-242ddaf0-kserve-87686648f-l9tjf" podUID="5fb7086a-96b1-4b1c-a86d-8e38bca4cf80" containerName="main" probeResult="failure" output="Get \"https://10.133.0.127:8000/ready\": context deadline exceeded"
Oct 30 22:12:53.552949 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:53.539487    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-dde1e8/distinct-w0-dde1e8-kserve-75b68b4c5d-hts2k" podUID="9d72e7c9-657e-4278-8ed9-53adb75bd530" containerName="main" probeResult="failure" output="Get \"https://10.133.0.72:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:12:53.732951 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:53.730038    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-29cbad/premium-w6-29cbad-kserve-6c9f8d6899-qtv6c" podUID="381b11f6-3fde-4de7-9ad3-7c0891e76ba1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.60:8000/ready\": context deadline exceeded"
Oct 30 22:12:53.894946 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:53.894036    4473 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-dde1e8/distinct-w0-dde1e8-kserve-75b68b4c5d-hts2k" podUID="9d72e7c9-657e-4278-8ed9-53adb75bd530" containerName="main" probeResult="failure" output="Get \"https://10.133.0.72:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:12:54.051444 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:54.049022    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-dde1e8/distinct-2-w0-dde1e8-kserve-744857d57-ftt8b" podUID="577b020d-2a6b-4020-863c-cf19d4f0ca3b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.79:8000/ready\": context deadline exceeded"
Oct 30 22:12:54.051444 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:54.049680    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-dde1e8/distinct-2-w0-dde1e8-kserve-744857d57-ftt8b"
Oct 30 22:12:54.167429 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:54.167357    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-0c99b1/distinct-2-w2-0c99b1-kserve-7466864658-8z7fk" podUID="89c09555-cf53-49a7-9c3b-95d2da2a293a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.82:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:12:54.194126 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:54.194043    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-0c99b1/distinct-2-w2-0c99b1-kserve-7466864658-8z7fk"
Oct 30 22:12:54.206089 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:54.197244    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-0f42cb/distinct-w3-0f42cb-kserve-fb54fdd59-n6w4l" podUID="ff3c2b25-08aa-4b70-bf57-9cf4a11d05ed" containerName="main" probeResult="failure" output="Get \"https://10.133.0.73:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:12:54.206089 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:54.197780    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-0f42cb/distinct-w3-0f42cb-kserve-fb54fdd59-n6w4l"
Oct 30 22:12:54.274968 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:54.272980    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-9f8aea/distinct-w5-9f8aea-kserve-5b564579cd-kmxzz" podUID="b11e8b11-334f-4a9e-98c2-b0164567a8af" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": context deadline exceeded"
Oct 30 22:12:54.274968 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:54.273515    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-9f8aea/distinct-w5-9f8aea-kserve-5b564579cd-kmxzz"
Oct 30 22:12:54.334991 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:54.323266    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-0f42cb/distinct-2-w3-0f42cb-kserve-fc477897b-jk8qc" podUID="61b49182-057e-41af-b609-1a6abf82c197" containerName="main" probeResult="failure" output="Get \"https://10.133.0.80:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:12:54.354947 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:54.339264    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-0f42cb/distinct-2-w3-0f42cb-kserve-fc477897b-jk8qc"
Oct 30 22:12:54.566985 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:54.538668    4473 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w6-0a7679-69cc484995-hlw4x" podStartSLOduration=5.538646158 podStartE2EDuration="5.538646158s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:12:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:12:52.397270745 +0000 UTC m=+1818.170639234" watchObservedRunningTime="2026-10-30 22:12:54.538646158 +0000 UTC m=+1820.312014646"
Oct 30 22:12:54.566985 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:54.548796    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-mt-242ddaf0/e2e-lifecycle-model-242ddaf0-kserve-87686648f-l9tjf" podUID="5fb7086a-96b1-4b1c-a86d-8e38bca4cf80" containerName="main" probeResult="failure" output="Get \"https://10.133.0.127:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:12:54.591525 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:54.591401    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-dde1e8/distinct-w0-dde1e8-kserve-75b68b4c5d-hts2k" podUID="9d72e7c9-657e-4278-8ed9-53adb75bd530" containerName="main" probeResult="failure" output="Get \"https://10.133.0.72:8000/ready\": context deadline exceeded"
Oct 30 22:12:54.799957 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:54.750149    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-29cbad/premium-w6-29cbad-kserve-6c9f8d6899-qtv6c" podUID="381b11f6-3fde-4de7-9ad3-7c0891e76ba1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:12:55.074986 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:55.053185    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-dde1e8/distinct-2-w0-dde1e8-kserve-744857d57-ftt8b" podUID="577b020d-2a6b-4020-863c-cf19d4f0ca3b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.79:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:12:55.186155 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:55.177769    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-0c99b1/distinct-2-w2-0c99b1-kserve-7466864658-8z7fk" podUID="89c09555-cf53-49a7-9c3b-95d2da2a293a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.82:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:12:55.313956 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:55.286556    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-9f8aea/distinct-w5-9f8aea-kserve-5b564579cd-kmxzz" podUID="b11e8b11-334f-4a9e-98c2-b0164567a8af" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:12:55.354014 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:55.325505    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-0f42cb/distinct-2-w3-0f42cb-kserve-fc477897b-jk8qc" podUID="61b49182-057e-41af-b609-1a6abf82c197" containerName="main" probeResult="failure" output="Get \"https://10.133.0.80:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:12:55.423049 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:12:55.422999    4473 kubelet.go:2711] "Housekeeping took longer than expected" err="housekeeping took too long" expected="1s" actual="1.041s"
Oct 30 22:12:55.423554 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:55.423530    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-242ddaf0/e2e-lifecycle-model-242ddaf0-kserve-87686648f-l9tjf"
Oct 30 22:12:55.423746 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:55.423734    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-dde1e8/distinct-w0-dde1e8-kserve-75b68b4c5d-hts2k"
Oct 30 22:12:55.424058 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:55.424042    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-29cbad/premium-w6-29cbad-kserve-6c9f8d6899-qtv6c"
Oct 30 22:12:55.424210 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:55.424198    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-0f42cb/distinct-w3-0f42cb-kserve-fb54fdd59-n6w4l"
Oct 30 22:12:55.424334 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:55.424323    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-29cbad/premium-w6-29cbad-kserve-6c9f8d6899-qtv6c"
Oct 30 22:12:55.553733 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:55.553684    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-mt-242ddaf0/e2e-lifecycle-model-242ddaf0-kserve-87686648f-l9tjf" podUID="5fb7086a-96b1-4b1c-a86d-8e38bca4cf80" containerName="main" probeResult="failure" output="Get \"https://10.133.0.127:8000/ready\": context deadline exceeded"
Oct 30 22:12:55.613186 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:55.613066    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-dde1e8/distinct-w0-dde1e8-kserve-75b68b4c5d-hts2k"
Oct 30 22:12:55.806380 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:55.806334    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-9f8aea/distinct-w5-9f8aea-kserve-5b564579cd-kmxzz"
Oct 30 22:12:55.975086 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:55.975049    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-dde1e8/distinct-2-w0-dde1e8-kserve-744857d57-ftt8b"
Oct 30 22:12:55.975300 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:55.975117    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-242ddaf0/e2e-lifecycle-model-242ddaf0-kserve-87686648f-l9tjf"
Oct 30 22:12:55.975300 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:55.975162    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-0f42cb/distinct-2-w3-0f42cb-kserve-fc477897b-jk8qc"
Oct 30 22:12:56.082553 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:12:56.082469    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-0c99b1/distinct-2-w2-0c99b1-kserve-7466864658-8z7fk"
Oct 30 22:13:03.169168 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:13:03.169087    4473 generic.go:356] "Generic (PLEG): container finished" podID="61782e91-1520-4b83-ad8d-8dd944881b52" containerID="414face9ec799d471c949aa7a6ab3c8d0c101d52e8254a13d48e49b791e7069e" exitCode=0
Oct 30 22:13:03.169666 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:13:03.169162    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w5-e431fc-openshift-default-6d78d86df7-lhwsl" event={"ID":"61782e91-1520-4b83-ad8d-8dd944881b52","Type":"ContainerDied","Data":"414face9ec799d471c949aa7a6ab3c8d0c101d52e8254a13d48e49b791e7069e"}
Oct 30 22:13:03.169727 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:13:03.169713    4473 scope.go:122] "RemoveContainer" containerID="414face9ec799d471c949aa7a6ab3c8d0c101d52e8254a13d48e49b791e7069e"
Oct 30 22:13:03.171381 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:13:03.171360    4473 generic.go:356] "Generic (PLEG): container finished" podID="c15267dd-1a03-40d6-b6b1-ae0dec8cc68f" containerID="39290de07fb2eaf514609b7a187a6af002d0fbd2a976ebcda2ba8f7cf3298592" exitCode=0
Oct 30 22:13:03.171498 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:13:03.171405    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w0-ebddcc-openshift-default-5d98df957-nmwbj" event={"ID":"c15267dd-1a03-40d6-b6b1-ae0dec8cc68f","Type":"ContainerDied","Data":"39290de07fb2eaf514609b7a187a6af002d0fbd2a976ebcda2ba8f7cf3298592"}
Oct 30 22:13:03.171673 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:13:03.171644    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:13:03.171730 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:13:03.171716    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:13:03.171766 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:13:03.171757    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:13:03.171799 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:13:03.171770    4473 scope.go:122] "RemoveContainer" containerID="39290de07fb2eaf514609b7a187a6af002d0fbd2a976ebcda2ba8f7cf3298592"
Oct 30 22:13:03.173528 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:13:03.173497    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:13:03.173599 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:13:03.173582    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:13:03.173646 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:13:03.173631    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:13:04.188623 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:13:04.188590    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w5-e431fc-openshift-default-6d78d86df7-lhwsl" event={"ID":"61782e91-1520-4b83-ad8d-8dd944881b52","Type":"ContainerStarted","Data":"507749823c9e799137e00bbb0c980cdf0023eb744fa7b5df0da742998c5ddc81"}
Oct 30 22:13:04.190743 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:13:04.190718    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w0-ebddcc-openshift-default-5d98df957-nmwbj" event={"ID":"c15267dd-1a03-40d6-b6b1-ae0dec8cc68f","Type":"ContainerStarted","Data":"edc445cee72a17ddcf070a36329eb7577a159ae6d37e69d89ad5e4e189335ea3"}
Oct 30 22:13:04.384229 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:13:04.384201    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w5-e431fc-openshift-default-6d78d86df7-lhwsl"
Oct 30 22:13:04.385179 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:13:04.385143    4473 patch_prober.go:29] interesting pod/e2e-shared-a-w5-e431fc-openshift-default-6d78d86df7-lhwsl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.122:15021/healthz/ready\": dial tcp 10.133.0.122:15021: connect: connection refused" start-of-body=
Oct 30 22:13:04.385325 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:13:04.385213    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w5-e431fc-openshift-default-6d78d86df7-lhwsl" podUID="61782e91-1520-4b83-ad8d-8dd944881b52" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.122:15021/healthz/ready\": dial tcp 10.133.0.122:15021: connect: connection refused"
Oct 30 22:13:05.108194 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:13:05.108166    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w6-0a7679-69cc484995-hlw4x"
Oct 30 22:13:05.113912 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:13:05.113892    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w0-ebddcc-openshift-default-5d98df957-nmwbj"
Oct 30 22:13:05.114082 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:13:05.114065    4473 patch_prober.go:29] interesting pod/e2e-shared-a-w0-ebddcc-openshift-default-5d98df957-nmwbj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.119:15021/healthz/ready\": dial tcp 10.133.0.119:15021: connect: connection refused" start-of-body=
Oct 30 22:13:05.114129 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:13:05.114117    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w0-ebddcc-openshift-default-5d98df957-nmwbj" podUID="c15267dd-1a03-40d6-b6b1-ae0dec8cc68f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.119:15021/healthz/ready\": dial tcp 10.133.0.119:15021: connect: connection refused"
Oct 30 22:13:05.386969 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:13:05.383948    4473 patch_prober.go:29] interesting pod/e2e-shared-a-w5-e431fc-openshift-default-6d78d86df7-lhwsl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.122:15021/healthz/ready\": dial tcp 10.133.0.122:15021: connect: connection refused" start-of-body=
Oct 30 22:13:05.386969 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:13:05.384046    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w5-e431fc-openshift-default-6d78d86df7-lhwsl" podUID="61782e91-1520-4b83-ad8d-8dd944881b52" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.122:15021/healthz/ready\": dial tcp 10.133.0.122:15021: connect: connection refused"
Oct 30 22:13:06.114255 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:13:06.114219    4473 patch_prober.go:29] interesting pod/e2e-shared-a-w0-ebddcc-openshift-default-5d98df957-nmwbj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.119:15021/healthz/ready\": dial tcp 10.133.0.119:15021: connect: connection refused" start-of-body=
Oct 30 22:13:06.114446 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:13:06.114273    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w0-ebddcc-openshift-default-5d98df957-nmwbj" podUID="c15267dd-1a03-40d6-b6b1-ae0dec8cc68f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.119:15021/healthz/ready\": dial tcp 10.133.0.119:15021: connect: connection refused"
Oct 30 22:13:06.383835 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:13:06.383752    4473 patch_prober.go:29] interesting pod/e2e-shared-a-w5-e431fc-openshift-default-6d78d86df7-lhwsl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.122:15021/healthz/ready\": dial tcp 10.133.0.122:15021: connect: connection refused" start-of-body=
Oct 30 22:13:06.383835 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:13:06.383798    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w5-e431fc-openshift-default-6d78d86df7-lhwsl" podUID="61782e91-1520-4b83-ad8d-8dd944881b52" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.122:15021/healthz/ready\": dial tcp 10.133.0.122:15021: connect: connection refused"
Oct 30 22:13:06.753011 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:13:06.752569    4473 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-242ddaf0-openshift-default-79b7ddb6f7-vxx64"]
Oct 30 22:13:06.781578 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:13:06.780434    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-242ddaf0-openshift-default-79b7ddb6f7-vxx64"
Oct 30 22:13:06.817386 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:13:06.817341    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/926d7274-b7f3-4a98-ab8b-c8dbe1178f36-credential-socket\") pod \"e2e-mt-242ddaf0-openshift-default-79b7ddb6f7-vxx64\" (UID: \"926d7274-b7f3-4a98-ab8b-c8dbe1178f36\") " pod="openshift-ingress/e2e-mt-242ddaf0-openshift-default-79b7ddb6f7-vxx64"
Oct 30 22:13:06.817576 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:13:06.817408    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rrknj\" (UniqueName: \"kubernetes.io/projected/926d7274-b7f3-4a98-ab8b-c8dbe1178f36-kube-api-access-rrknj\") pod \"e2e-mt-242ddaf0-openshift-default-79b7ddb6f7-vxx64\" (UID: \"926d7274-b7f3-4a98-ab8b-c8dbe1178f36\") " pod="openshift-ingress/e2e-mt-242ddaf0-openshift-default-79b7ddb6f7-vxx64"
Oct 30 22:13:06.817624 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:13:06.817588    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/926d7274-b7f3-4a98-ab8b-c8dbe1178f36-istio-data\") pod \"e2e-mt-242ddaf0-openshift-default-79b7ddb6f7-vxx64\" (UID: \"926d7274-b7f3-4a98-ab8b-c8dbe1178f36\") " pod="openshift-ingress/e2e-mt-242ddaf0-openshift-default-79b7ddb6f7-vxx64"
Oct 30 22:13:06.817691 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:13:06.817654    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/926d7274-b7f3-4a98-ab8b-c8dbe1178f36-istio-envoy\") pod \"e2e-mt-242ddaf0-openshift-default-79b7ddb6f7-vxx64\" (UID: \"926d7274-b7f3-4a98-ab8b-c8dbe1178f36\") " pod="openshift-ingress/e2e-mt-242ddaf0-openshift-default-79b7ddb6f7-vxx64"
Oct 30 22:13:06.817786 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:13:06.817764    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/926d7274-b7f3-4a98-ab8b-c8dbe1178f36-istiod-ca-cert\") pod \"e2e-mt-242ddaf0-openshift-default-79b7ddb6f7-vxx64\" (UID: \"926d7274-b7f3-4a98-ab8b-c8dbe1178f36\") " pod="openshift-ingress/e2e-mt-242ddaf0-openshift-default-79b7ddb6f7-vxx64"
Oct 30 22:13:06.817830 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:13:06.817820    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/926d7274-b7f3-4a98-ab8b-c8dbe1178f36-workload-certs\") pod \"e2e-mt-242ddaf0-openshift-default-79b7ddb6f7-vxx64\" (UID: \"926d7274-b7f3-4a98-ab8b-c8dbe1178f36\") " pod="openshift-ingress/e2e-mt-242ddaf0-openshift-default-79b7ddb6f7-vxx64"
Oct 30 22:13:06.817956 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:13:06.817933    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/926d7274-b7f3-4a98-ab8b-c8dbe1178f36-istio-podinfo\") pod \"e2e-mt-242ddaf0-openshift-default-79b7ddb6f7-vxx64\" (UID: \"926d7274-b7f3-4a98-ab8b-c8dbe1178f36\") " pod="openshift-ingress/e2e-mt-242ddaf0-openshift-default-79b7ddb6f7-vxx64"
Oct 30 22:13:06.818054 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:13:06.818036    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/926d7274-b7f3-4a98-ab8b-c8dbe1178f36-istio-token\") pod \"e2e-mt-242ddaf0-openshift-default-79b7ddb6f7-vxx64\" (UID: \"926d7274-b7f3-4a98-ab8b-c8dbe1178f36\") " pod="openshift-ingress/e2e-mt-242ddaf0-openshift-default-79b7ddb6f7-vxx64"
Oct 30 22:13:06.818158 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:13:06.818140    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/926d7274-b7f3-4a98-ab8b-c8dbe1178f36-workload-socket\") pod \"e2e-mt-242ddaf0-openshift-default-79b7ddb6f7-vxx64\" (UID: \"926d7274-b7f3-4a98-ab8b-c8dbe1178f36\") " pod="openshift-ingress/e2e-mt-242ddaf0-openshift-default-79b7ddb6f7-vxx64"
Oct 30 22:13:06.859953 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:13:06.859696    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-242ddaf0-openshift-default-dockercfg-ljxqg\""
Oct 30 22:13:06.898080 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:13:06.895223    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-242ddaf0-openshift-default-79b7ddb6f7-vxx64"]
Oct 30 22:13:06.922449 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:13:06.922408    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/926d7274-b7f3-4a98-ab8b-c8dbe1178f36-istio-token\") pod \"e2e-mt-242ddaf0-openshift-default-79b7ddb6f7-vxx64\" (UID: \"926d7274-b7f3-4a98-ab8b-c8dbe1178f36\") " pod="openshift-ingress/e2e-mt-242ddaf0-openshift-default-79b7ddb6f7-vxx64"
Oct 30 22:13:06.922637 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:13:06.922500    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/926d7274-b7f3-4a98-ab8b-c8dbe1178f36-workload-socket\") pod \"e2e-mt-242ddaf0-openshift-default-79b7ddb6f7-vxx64\" (UID: \"926d7274-b7f3-4a98-ab8b-c8dbe1178f36\") " pod="openshift-ingress/e2e-mt-242ddaf0-openshift-default-79b7ddb6f7-vxx64"
Oct 30 22:13:06.922637 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:13:06.922563    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/926d7274-b7f3-4a98-ab8b-c8dbe1178f36-credential-socket\") pod \"e2e-mt-242ddaf0-openshift-default-79b7ddb6f7-vxx64\" (UID: \"926d7274-b7f3-4a98-ab8b-c8dbe1178f36\") " pod="openshift-ingress/e2e-mt-242ddaf0-openshift-default-79b7ddb6f7-vxx64"
Oct 30 22:13:06.922637 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:13:06.922604    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rrknj\" (UniqueName: \"kubernetes.io/projected/926d7274-b7f3-4a98-ab8b-c8dbe1178f36-kube-api-access-rrknj\") pod \"e2e-mt-242ddaf0-openshift-default-79b7ddb6f7-vxx64\" (UID: \"926d7274-b7f3-4a98-ab8b-c8dbe1178f36\") " pod="openshift-ingress/e2e-mt-242ddaf0-openshift-default-79b7ddb6f7-vxx64"
Oct 30 22:13:06.922760 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:13:06.922691    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/926d7274-b7f3-4a98-ab8b-c8dbe1178f36-istio-data\") pod \"e2e-mt-242ddaf0-openshift-default-79b7ddb6f7-vxx64\" (UID: \"926d7274-b7f3-4a98-ab8b-c8dbe1178f36\") " pod="openshift-ingress/e2e-mt-242ddaf0-openshift-default-79b7ddb6f7-vxx64"
Oct 30 22:13:06.922760 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:13:06.922716    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/926d7274-b7f3-4a98-ab8b-c8dbe1178f36-istio-envoy\") pod \"e2e-mt-242ddaf0-openshift-default-79b7ddb6f7-vxx64\" (UID: \"926d7274-b7f3-4a98-ab8b-c8dbe1178f36\") " pod="openshift-ingress/e2e-mt-242ddaf0-openshift-default-79b7ddb6f7-vxx64"
Oct 30 22:13:06.922842 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:13:06.922792    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/926d7274-b7f3-4a98-ab8b-c8dbe1178f36-istiod-ca-cert\") pod \"e2e-mt-242ddaf0-openshift-default-79b7ddb6f7-vxx64\" (UID: \"926d7274-b7f3-4a98-ab8b-c8dbe1178f36\") " pod="openshift-ingress/e2e-mt-242ddaf0-openshift-default-79b7ddb6f7-vxx64"
Oct 30 22:13:06.922842 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:13:06.922824    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/926d7274-b7f3-4a98-ab8b-c8dbe1178f36-workload-certs\") pod \"e2e-mt-242ddaf0-openshift-default-79b7ddb6f7-vxx64\" (UID: \"926d7274-b7f3-4a98-ab8b-c8dbe1178f36\") " pod="openshift-ingress/e2e-mt-242ddaf0-openshift-default-79b7ddb6f7-vxx64"
Oct 30 22:13:06.923167 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:13:06.923144    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/926d7274-b7f3-4a98-ab8b-c8dbe1178f36-istio-podinfo\") pod \"e2e-mt-242ddaf0-openshift-default-79b7ddb6f7-vxx64\" (UID: \"926d7274-b7f3-4a98-ab8b-c8dbe1178f36\") " pod="openshift-ingress/e2e-mt-242ddaf0-openshift-default-79b7ddb6f7-vxx64"
Oct 30 22:13:06.924621 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:13:06.924367    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/926d7274-b7f3-4a98-ab8b-c8dbe1178f36-workload-socket\") pod \"e2e-mt-242ddaf0-openshift-default-79b7ddb6f7-vxx64\" (UID: \"926d7274-b7f3-4a98-ab8b-c8dbe1178f36\") " pod="openshift-ingress/e2e-mt-242ddaf0-openshift-default-79b7ddb6f7-vxx64"
Oct 30 22:13:06.924621 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:13:06.924506    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/926d7274-b7f3-4a98-ab8b-c8dbe1178f36-workload-certs\") pod \"e2e-mt-242ddaf0-openshift-default-79b7ddb6f7-vxx64\" (UID: \"926d7274-b7f3-4a98-ab8b-c8dbe1178f36\") " pod="openshift-ingress/e2e-mt-242ddaf0-openshift-default-79b7ddb6f7-vxx64"
Oct 30 22:13:06.924838 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:13:06.924624    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/926d7274-b7f3-4a98-ab8b-c8dbe1178f36-credential-socket\") pod \"e2e-mt-242ddaf0-openshift-default-79b7ddb6f7-vxx64\" (UID: \"926d7274-b7f3-4a98-ab8b-c8dbe1178f36\") " pod="openshift-ingress/e2e-mt-242ddaf0-openshift-default-79b7ddb6f7-vxx64"
Oct 30 22:13:06.933232 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:13:06.925096    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/926d7274-b7f3-4a98-ab8b-c8dbe1178f36-istiod-ca-cert\") pod \"e2e-mt-242ddaf0-openshift-default-79b7ddb6f7-vxx64\" (UID: \"926d7274-b7f3-4a98-ab8b-c8dbe1178f36\") " pod="openshift-ingress/e2e-mt-242ddaf0-openshift-default-79b7ddb6f7-vxx64"
Oct 30 22:13:06.988198 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:13:06.988161    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/926d7274-b7f3-4a98-ab8b-c8dbe1178f36-istio-envoy\") pod \"e2e-mt-242ddaf0-openshift-default-79b7ddb6f7-vxx64\" (UID: \"926d7274-b7f3-4a98-ab8b-c8dbe1178f36\") " pod="openshift-ingress/e2e-mt-242ddaf0-openshift-default-79b7ddb6f7-vxx64"
Oct 30 22:13:06.992104 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:13:06.992074    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/926d7274-b7f3-4a98-ab8b-c8dbe1178f36-istio-podinfo\") pod \"e2e-mt-242ddaf0-openshift-default-79b7ddb6f7-vxx64\" (UID: \"926d7274-b7f3-4a98-ab8b-c8dbe1178f36\") " pod="openshift-ingress/e2e-mt-242ddaf0-openshift-default-79b7ddb6f7-vxx64"
Oct 30 22:13:06.993148 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:13:06.993112    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/926d7274-b7f3-4a98-ab8b-c8dbe1178f36-istio-data\") pod \"e2e-mt-242ddaf0-openshift-default-79b7ddb6f7-vxx64\" (UID: \"926d7274-b7f3-4a98-ab8b-c8dbe1178f36\") " pod="openshift-ingress/e2e-mt-242ddaf0-openshift-default-79b7ddb6f7-vxx64"
Oct 30 22:13:06.996670 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:13:06.996610    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rrknj\" (UniqueName: \"kubernetes.io/projected/926d7274-b7f3-4a98-ab8b-c8dbe1178f36-kube-api-access-rrknj\") pod \"e2e-mt-242ddaf0-openshift-default-79b7ddb6f7-vxx64\" (UID: \"926d7274-b7f3-4a98-ab8b-c8dbe1178f36\") " pod="openshift-ingress/e2e-mt-242ddaf0-openshift-default-79b7ddb6f7-vxx64"
Oct 30 22:13:07.002370 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:13:07.002337    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/926d7274-b7f3-4a98-ab8b-c8dbe1178f36-istio-token\") pod \"e2e-mt-242ddaf0-openshift-default-79b7ddb6f7-vxx64\" (UID: \"926d7274-b7f3-4a98-ab8b-c8dbe1178f36\") " pod="openshift-ingress/e2e-mt-242ddaf0-openshift-default-79b7ddb6f7-vxx64"
Oct 30 22:13:07.095971 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:13:07.095868    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-242ddaf0-openshift-default-79b7ddb6f7-vxx64"
Oct 30 22:13:07.114821 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:13:07.114372    4473 patch_prober.go:29] interesting pod/e2e-shared-a-w0-ebddcc-openshift-default-5d98df957-nmwbj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.119:15021/healthz/ready\": dial tcp 10.133.0.119:15021: connect: connection refused" start-of-body=
Oct 30 22:13:07.114821 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:13:07.114431    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w0-ebddcc-openshift-default-5d98df957-nmwbj" podUID="c15267dd-1a03-40d6-b6b1-ae0dec8cc68f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.119:15021/healthz/ready\": dial tcp 10.133.0.119:15021: connect: connection refused"
Oct 30 22:13:07.383490 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:13:07.383456    4473 patch_prober.go:29] interesting pod/e2e-shared-a-w5-e431fc-openshift-default-6d78d86df7-lhwsl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.122:15021/healthz/ready\": dial tcp 10.133.0.122:15021: connect: connection refused" start-of-body=
Oct 30 22:13:07.383698 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:13:07.383520    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w5-e431fc-openshift-default-6d78d86df7-lhwsl" podUID="61782e91-1520-4b83-ad8d-8dd944881b52" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.122:15021/healthz/ready\": dial tcp 10.133.0.122:15021: connect: connection refused"
Oct 30 22:13:07.680693 ip-10-0-140-18 kubenswrapper[4473]: W1030 22:13:07.679631    4473 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod926d7274_b7f3_4a98_ab8b_c8dbe1178f36.slice/crio-3ceff137d44b351e22010d09b05192ce61f495140a48e618e3410ee464b8c429 WatchSource:0}: Error finding container 3ceff137d44b351e22010d09b05192ce61f495140a48e618e3410ee464b8c429: Status 404 returned error can't find the container with id 3ceff137d44b351e22010d09b05192ce61f495140a48e618e3410ee464b8c429
Oct 30 22:13:07.681683 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:13:07.681162    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-242ddaf0-openshift-default-79b7ddb6f7-vxx64"]
Oct 30 22:13:07.684436 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:13:07.684400    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:13:07.684556 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:13:07.684477    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:13:07.684556 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:13:07.684508    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:13:08.114350 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:13:08.114310    4473 patch_prober.go:29] interesting pod/e2e-shared-a-w0-ebddcc-openshift-default-5d98df957-nmwbj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.119:15021/healthz/ready\": dial tcp 10.133.0.119:15021: connect: connection refused" start-of-body=
Oct 30 22:13:08.114852 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:13:08.114362    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w0-ebddcc-openshift-default-5d98df957-nmwbj" podUID="c15267dd-1a03-40d6-b6b1-ae0dec8cc68f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.119:15021/healthz/ready\": dial tcp 10.133.0.119:15021: connect: connection refused"
Oct 30 22:13:08.251640 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:13:08.250403    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-242ddaf0-openshift-default-79b7ddb6f7-vxx64" event={"ID":"926d7274-b7f3-4a98-ab8b-c8dbe1178f36","Type":"ContainerStarted","Data":"c49c685014bb88936bdd56ed7f19bdec39e10af6dd170e2fcd7ad86157cf58ae"}
Oct 30 22:13:08.251640 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:13:08.250474    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-242ddaf0-openshift-default-79b7ddb6f7-vxx64" event={"ID":"926d7274-b7f3-4a98-ab8b-c8dbe1178f36","Type":"ContainerStarted","Data":"3ceff137d44b351e22010d09b05192ce61f495140a48e618e3410ee464b8c429"}
Oct 30 22:13:08.382655 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:13:08.381747    4473 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-242ddaf0-openshift-default-79b7ddb6f7-vxx64" podStartSLOduration=2.381727529 podStartE2EDuration="2.381727529s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:13:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:13:08.381563241 +0000 UTC m=+1834.154931740" watchObservedRunningTime="2026-10-30 22:13:08.381727529 +0000 UTC m=+1834.155096017"
Oct 30 22:13:08.383770 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:13:08.383741    4473 patch_prober.go:29] interesting pod/e2e-shared-a-w5-e431fc-openshift-default-6d78d86df7-lhwsl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.122:15021/healthz/ready\": dial tcp 10.133.0.122:15021: connect: connection refused" start-of-body=
Oct 30 22:13:08.383973 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:13:08.383785    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w5-e431fc-openshift-default-6d78d86df7-lhwsl" podUID="61782e91-1520-4b83-ad8d-8dd944881b52" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.122:15021/healthz/ready\": dial tcp 10.133.0.122:15021: connect: connection refused"
Oct 30 22:13:08.976948 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:13:08.976755    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w5-e431fc-openshift-default-6d78d86df7-lhwsl"
Oct 30 22:13:09.102945 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:13:09.101998    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-242ddaf0-openshift-default-79b7ddb6f7-vxx64"
Oct 30 22:13:09.105142 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:13:09.105040    4473 patch_prober.go:29] interesting pod/e2e-mt-242ddaf0-openshift-default-79b7ddb6f7-vxx64 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.133:15021/healthz/ready\": dial tcp 10.133.0.133:15021: connect: connection refused" start-of-body=
Oct 30 22:13:09.105142 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:13:09.105113    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-242ddaf0-openshift-default-79b7ddb6f7-vxx64" podUID="926d7274-b7f3-4a98-ab8b-c8dbe1178f36" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.133:15021/healthz/ready\": dial tcp 10.133.0.133:15021: connect: connection refused"
Oct 30 22:13:09.113794 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:13:09.113759    4473 patch_prober.go:29] interesting pod/e2e-shared-a-w0-ebddcc-openshift-default-5d98df957-nmwbj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.119:15021/healthz/ready\": dial tcp 10.133.0.119:15021: connect: connection refused" start-of-body=
Oct 30 22:13:09.113963 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:13:09.113810    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w0-ebddcc-openshift-default-5d98df957-nmwbj" podUID="c15267dd-1a03-40d6-b6b1-ae0dec8cc68f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.119:15021/healthz/ready\": dial tcp 10.133.0.119:15021: connect: connection refused"
Oct 30 22:13:09.412984 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:13:09.411454    4473 patch_prober.go:29] interesting pod/e2e-shared-a-w5-e431fc-openshift-default-6d78d86df7-lhwsl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.122:15021/healthz/ready\": dial tcp 10.133.0.122:15021: connect: connection refused" start-of-body=
Oct 30 22:13:09.412984 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:13:09.411513    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w5-e431fc-openshift-default-6d78d86df7-lhwsl" podUID="61782e91-1520-4b83-ad8d-8dd944881b52" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.122:15021/healthz/ready\": dial tcp 10.133.0.122:15021: connect: connection refused"
Oct 30 22:13:10.097465 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:13:10.097416    4473 patch_prober.go:29] interesting pod/e2e-mt-242ddaf0-openshift-default-79b7ddb6f7-vxx64 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.133:15021/healthz/ready\": dial tcp 10.133.0.133:15021: connect: connection refused" start-of-body=
Oct 30 22:13:10.097654 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:13:10.097493    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-242ddaf0-openshift-default-79b7ddb6f7-vxx64" podUID="926d7274-b7f3-4a98-ab8b-c8dbe1178f36" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.133:15021/healthz/ready\": dial tcp 10.133.0.133:15021: connect: connection refused"
Oct 30 22:13:10.119500 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:13:10.119469    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w0-ebddcc-openshift-default-5d98df957-nmwbj"
Oct 30 22:13:10.119980 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:13:10.119960    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w0-ebddcc-openshift-default-5d98df957-nmwbj"
Oct 30 22:13:10.120834 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:13:10.120805    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w0-ebddcc-openshift-default-5d98df957-nmwbj"
Oct 30 22:13:10.388778 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:13:10.388749    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w5-e431fc-openshift-default-6d78d86df7-lhwsl"
Oct 30 22:13:10.390137 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:13:10.390113    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w5-e431fc-openshift-default-6d78d86df7-lhwsl"
Oct 30 22:13:11.097454 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:13:11.097416    4473 patch_prober.go:29] interesting pod/e2e-mt-242ddaf0-openshift-default-79b7ddb6f7-vxx64 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.133:15021/healthz/ready\": dial tcp 10.133.0.133:15021: connect: connection refused" start-of-body=
Oct 30 22:13:11.097850 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:13:11.097472    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-242ddaf0-openshift-default-79b7ddb6f7-vxx64" podUID="926d7274-b7f3-4a98-ab8b-c8dbe1178f36" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.133:15021/healthz/ready\": dial tcp 10.133.0.133:15021: connect: connection refused"
Oct 30 22:13:12.097218 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:13:12.097178    4473 patch_prober.go:29] interesting pod/e2e-mt-242ddaf0-openshift-default-79b7ddb6f7-vxx64 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.133:15021/healthz/ready\": dial tcp 10.133.0.133:15021: connect: connection refused" start-of-body=
Oct 30 22:13:12.097435 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:13:12.097243    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-242ddaf0-openshift-default-79b7ddb6f7-vxx64" podUID="926d7274-b7f3-4a98-ab8b-c8dbe1178f36" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.133:15021/healthz/ready\": dial tcp 10.133.0.133:15021: connect: connection refused"
Oct 30 22:13:13.108393 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:13:13.102689    4473 patch_prober.go:29] interesting pod/e2e-mt-242ddaf0-openshift-default-79b7ddb6f7-vxx64 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.133:15021/healthz/ready\": dial tcp 10.133.0.133:15021: connect: connection refused" start-of-body=
Oct 30 22:13:13.108393 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:13:13.107833    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-242ddaf0-openshift-default-79b7ddb6f7-vxx64" podUID="926d7274-b7f3-4a98-ab8b-c8dbe1178f36" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.133:15021/healthz/ready\": dial tcp 10.133.0.133:15021: connect: connection refused"
Oct 30 22:13:14.097137 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:13:14.097094    4473 patch_prober.go:29] interesting pod/e2e-mt-242ddaf0-openshift-default-79b7ddb6f7-vxx64 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.133:15021/healthz/ready\": dial tcp 10.133.0.133:15021: connect: connection refused" start-of-body=
Oct 30 22:13:14.097323 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:13:14.097164    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-242ddaf0-openshift-default-79b7ddb6f7-vxx64" podUID="926d7274-b7f3-4a98-ab8b-c8dbe1178f36" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.133:15021/healthz/ready\": dial tcp 10.133.0.133:15021: connect: connection refused"
Oct 30 22:13:15.102170 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:13:15.102089    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-242ddaf0-openshift-default-79b7ddb6f7-vxx64"
Oct 30 22:13:15.102661 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:13:15.102611    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-242ddaf0-openshift-default-79b7ddb6f7-vxx64"
Oct 30 22:13:15.103765 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:13:15.103741    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-242ddaf0-openshift-default-79b7ddb6f7-vxx64"
Oct 30 22:13:20.327515 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:13:20.327476    4473 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fncvg" containerName="extractor"
Oct 30 22:13:50.403405 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:13:50.403366    4473 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fncvg" containerName="extractor"
Oct 30 22:14:10.378599 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:14:10.378567    4473 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w5-5dff71-7b4c67c84c-xs9qg"]
Oct 30 22:14:10.382533 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:14:10.382507    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w5-5dff71-7b4c67c84c-xs9qg"
Oct 30 22:14:10.412497 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:14:10.412453    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w5-5dff71-7b4c67c84c-xs9qg"]
Oct 30 22:14:10.457757 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:14:10.457720    4473 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w5-5dff71\""
Oct 30 22:14:10.457995 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:14:10.457966    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w5-5dff71-dockercfg-wnj4j\""
Oct 30 22:14:10.534837 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:14:10.534798    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hvxlk\" (UniqueName: \"kubernetes.io/projected/8bdfde4a-7366-46fd-97c6-6c529c9ce2a3-kube-api-access-hvxlk\") pod \"payload-pre-processing-e2e-shared-b-w5-5dff71-7b4c67c84c-xs9qg\" (UID: \"8bdfde4a-7366-46fd-97c6-6c529c9ce2a3\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w5-5dff71-7b4c67c84c-xs9qg"
Oct 30 22:14:10.535077 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:14:10.534969    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8bdfde4a-7366-46fd-97c6-6c529c9ce2a3-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w5-5dff71-7b4c67c84c-xs9qg\" (UID: \"8bdfde4a-7366-46fd-97c6-6c529c9ce2a3\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w5-5dff71-7b4c67c84c-xs9qg"
Oct 30 22:14:10.637068 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:14:10.636980    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hvxlk\" (UniqueName: \"kubernetes.io/projected/8bdfde4a-7366-46fd-97c6-6c529c9ce2a3-kube-api-access-hvxlk\") pod \"payload-pre-processing-e2e-shared-b-w5-5dff71-7b4c67c84c-xs9qg\" (UID: \"8bdfde4a-7366-46fd-97c6-6c529c9ce2a3\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w5-5dff71-7b4c67c84c-xs9qg"
Oct 30 22:14:10.637228 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:14:10.637082    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8bdfde4a-7366-46fd-97c6-6c529c9ce2a3-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w5-5dff71-7b4c67c84c-xs9qg\" (UID: \"8bdfde4a-7366-46fd-97c6-6c529c9ce2a3\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w5-5dff71-7b4c67c84c-xs9qg"
Oct 30 22:14:10.637750 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:14:10.637731    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8bdfde4a-7366-46fd-97c6-6c529c9ce2a3-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w5-5dff71-7b4c67c84c-xs9qg\" (UID: \"8bdfde4a-7366-46fd-97c6-6c529c9ce2a3\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w5-5dff71-7b4c67c84c-xs9qg"
Oct 30 22:14:10.695151 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:14:10.695117    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hvxlk\" (UniqueName: \"kubernetes.io/projected/8bdfde4a-7366-46fd-97c6-6c529c9ce2a3-kube-api-access-hvxlk\") pod \"payload-pre-processing-e2e-shared-b-w5-5dff71-7b4c67c84c-xs9qg\" (UID: \"8bdfde4a-7366-46fd-97c6-6c529c9ce2a3\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w5-5dff71-7b4c67c84c-xs9qg"
Oct 30 22:14:10.702095 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:14:10.702066    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w5-5dff71-7b4c67c84c-xs9qg"
Oct 30 22:14:10.813003 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:14:10.812962    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-2a25a4-75b5bd6c5-8fcr9"]
Oct 30 22:14:11.087138 ip-10-0-140-18 kubenswrapper[4473]: W1030 22:14:11.087107    4473 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8bdfde4a_7366_46fd_97c6_6c529c9ce2a3.slice/crio-8c0c382309db4c2441dca38a9556f9f7508a2a082325a91a32bb6933fa33071a WatchSource:0}: Error finding container 8c0c382309db4c2441dca38a9556f9f7508a2a082325a91a32bb6933fa33071a: Status 404 returned error can't find the container with id 8c0c382309db4c2441dca38a9556f9f7508a2a082325a91a32bb6933fa33071a
Oct 30 22:14:11.087310 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:14:11.087191    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w5-5dff71-7b4c67c84c-xs9qg"]
Oct 30 22:14:11.953014 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:14:11.952972    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w5-5dff71-7b4c67c84c-xs9qg" event={"ID":"8bdfde4a-7366-46fd-97c6-6c529c9ce2a3","Type":"ContainerStarted","Data":"f3f9a29e2d0de8c767849797a860f4090da88b8631a0fdf0348916a47410da24"}
Oct 30 22:14:11.953507 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:14:11.953074    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w5-5dff71-7b4c67c84c-xs9qg" event={"ID":"8bdfde4a-7366-46fd-97c6-6c529c9ce2a3","Type":"ContainerStarted","Data":"8c0c382309db4c2441dca38a9556f9f7508a2a082325a91a32bb6933fa33071a"}
Oct 30 22:14:11.954290 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:14:11.954263    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w5-5dff71-7b4c67c84c-xs9qg"
Oct 30 22:14:12.069441 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:14:12.069389    4473 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w5-5dff71-7b4c67c84c-xs9qg" podStartSLOduration=2.06937594 podStartE2EDuration="2.06937594s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:14:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:14:12.068222207 +0000 UTC m=+1897.841590753" watchObservedRunningTime="2026-10-30 22:14:12.06937594 +0000 UTC m=+1897.842744428"
Oct 30 22:14:12.451945 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:14:12.451056    4473 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-0a7679-745b8695b5-jlj2t"]
Oct 30 22:14:12.485959 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:14:12.483121    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-0a7679-745b8695b5-jlj2t"]
Oct 30 22:14:12.485959 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:14:12.483249    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-0a7679-745b8695b5-jlj2t"
Oct 30 22:14:12.522002 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:14:12.506534    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a9e13de9-d3e4-4200-9ac9-6e2ba583b818-metrics-certs\") pod \"maas-api-e2e-shared-b-w6-0a7679-745b8695b5-jlj2t\" (UID: \"a9e13de9-d3e4-4200-9ac9-6e2ba583b818\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-0a7679-745b8695b5-jlj2t"
Oct 30 22:14:12.522343 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:14:12.522307    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a9e13de9-d3e4-4200-9ac9-6e2ba583b818-maas-api-tls\") pod \"maas-api-e2e-shared-b-w6-0a7679-745b8695b5-jlj2t\" (UID: \"a9e13de9-d3e4-4200-9ac9-6e2ba583b818\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-0a7679-745b8695b5-jlj2t"
Oct 30 22:14:12.525950 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:14:12.522541    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nvcpl\" (UniqueName: \"kubernetes.io/projected/a9e13de9-d3e4-4200-9ac9-6e2ba583b818-kube-api-access-nvcpl\") pod \"maas-api-e2e-shared-b-w6-0a7679-745b8695b5-jlj2t\" (UID: \"a9e13de9-d3e4-4200-9ac9-6e2ba583b818\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-0a7679-745b8695b5-jlj2t"
Oct 30 22:14:12.546093 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:14:12.526300    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a9e13de9-d3e4-4200-9ac9-6e2ba583b818-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w6-0a7679-745b8695b5-jlj2t\" (UID: \"a9e13de9-d3e4-4200-9ac9-6e2ba583b818\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-0a7679-745b8695b5-jlj2t"
Oct 30 22:14:12.547311 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:14:12.537196    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w6-0a7679-serving-cert\""
Oct 30 22:14:12.736585 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:14:12.713825    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a9e13de9-d3e4-4200-9ac9-6e2ba583b818-maas-api-tls\") pod \"maas-api-e2e-shared-b-w6-0a7679-745b8695b5-jlj2t\" (UID: \"a9e13de9-d3e4-4200-9ac9-6e2ba583b818\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-0a7679-745b8695b5-jlj2t"
Oct 30 22:14:12.736585 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:14:12.713945    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nvcpl\" (UniqueName: \"kubernetes.io/projected/a9e13de9-d3e4-4200-9ac9-6e2ba583b818-kube-api-access-nvcpl\") pod \"maas-api-e2e-shared-b-w6-0a7679-745b8695b5-jlj2t\" (UID: \"a9e13de9-d3e4-4200-9ac9-6e2ba583b818\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-0a7679-745b8695b5-jlj2t"
Oct 30 22:14:12.736585 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:14:12.714041    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a9e13de9-d3e4-4200-9ac9-6e2ba583b818-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w6-0a7679-745b8695b5-jlj2t\" (UID: \"a9e13de9-d3e4-4200-9ac9-6e2ba583b818\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-0a7679-745b8695b5-jlj2t"
Oct 30 22:14:12.736585 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:14:12.714218    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a9e13de9-d3e4-4200-9ac9-6e2ba583b818-metrics-certs\") pod \"maas-api-e2e-shared-b-w6-0a7679-745b8695b5-jlj2t\" (UID: \"a9e13de9-d3e4-4200-9ac9-6e2ba583b818\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-0a7679-745b8695b5-jlj2t"
Oct 30 22:14:12.736585 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:14:12.722754    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a9e13de9-d3e4-4200-9ac9-6e2ba583b818-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w6-0a7679-745b8695b5-jlj2t\" (UID: \"a9e13de9-d3e4-4200-9ac9-6e2ba583b818\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-0a7679-745b8695b5-jlj2t"
Oct 30 22:14:12.908948 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:14:12.904400    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nvcpl\" (UniqueName: \"kubernetes.io/projected/a9e13de9-d3e4-4200-9ac9-6e2ba583b818-kube-api-access-nvcpl\") pod \"maas-api-e2e-shared-b-w6-0a7679-745b8695b5-jlj2t\" (UID: \"a9e13de9-d3e4-4200-9ac9-6e2ba583b818\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-0a7679-745b8695b5-jlj2t"
Oct 30 22:14:12.960969 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:14:12.958490    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a9e13de9-d3e4-4200-9ac9-6e2ba583b818-metrics-certs\") pod \"maas-api-e2e-shared-b-w6-0a7679-745b8695b5-jlj2t\" (UID: \"a9e13de9-d3e4-4200-9ac9-6e2ba583b818\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-0a7679-745b8695b5-jlj2t"
Oct 30 22:14:12.977283 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:14:12.962450    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a9e13de9-d3e4-4200-9ac9-6e2ba583b818-maas-api-tls\") pod \"maas-api-e2e-shared-b-w6-0a7679-745b8695b5-jlj2t\" (UID: \"a9e13de9-d3e4-4200-9ac9-6e2ba583b818\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-0a7679-745b8695b5-jlj2t"
Oct 30 22:14:13.018227 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:14:13.017884    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-0a7679-745b8695b5-jlj2t"
Oct 30 22:14:13.478359 ip-10-0-140-18 kubenswrapper[4473]: W1030 22:14:13.478329    4473 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9e13de9_d3e4_4200_9ac9_6e2ba583b818.slice/crio-56d8381ed595c4cccc07c082999f0bdf6a74851033cec7bee9e392efc8ded0e1 WatchSource:0}: Error finding container 56d8381ed595c4cccc07c082999f0bdf6a74851033cec7bee9e392efc8ded0e1: Status 404 returned error can't find the container with id 56d8381ed595c4cccc07c082999f0bdf6a74851033cec7bee9e392efc8ded0e1
Oct 30 22:14:13.478818 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:14:13.478796    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-0a7679-745b8695b5-jlj2t"]
Oct 30 22:14:13.978621 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:14:13.978579    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-0a7679-745b8695b5-jlj2t" event={"ID":"a9e13de9-d3e4-4200-9ac9-6e2ba583b818","Type":"ContainerStarted","Data":"56d8381ed595c4cccc07c082999f0bdf6a74851033cec7bee9e392efc8ded0e1"}
Oct 30 22:14:14.988829 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:14:14.988795    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-0a7679-745b8695b5-jlj2t" event={"ID":"a9e13de9-d3e4-4200-9ac9-6e2ba583b818","Type":"ContainerStarted","Data":"791c82775545365f5bd15ff8879ffad9cf414be059e508fd5ac9adf095db17f9"}
Oct 30 22:14:14.989379 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:14:14.988981    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-0a7679-745b8695b5-jlj2t"
Oct 30 22:14:15.111433 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:14:15.111383    4473 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-0a7679-745b8695b5-jlj2t" podStartSLOduration=2.172340741 podStartE2EDuration="3.111366934s" totalImagesPullingTime="939.026193ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:14:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:14:15.108157057 +0000 UTC m=+1900.881525545" watchObservedRunningTime="2026-10-30 22:14:15.111366934 +0000 UTC m=+1900.884735420"
Oct 30 22:14:15.813942 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:14:15.813878    4473 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-2a25a4-75b5bd6c5-8fcr9" podUID="050a6565-0515-4f27-8110-51262e515fa2" containerName="payload-processing" containerID="cri-o://44f32e02c30bb001e535b9e164041d3f08d03267c615524eb11b06c187bfaca1" gracePeriod=25
Oct 30 22:14:15.999088 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:14:15.999054    4473 generic.go:356] "Generic (PLEG): container finished" podID="050a6565-0515-4f27-8110-51262e515fa2" containerID="44f32e02c30bb001e535b9e164041d3f08d03267c615524eb11b06c187bfaca1" exitCode=0
Oct 30 22:14:15.999508 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:14:15.999138    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-2a25a4-75b5bd6c5-8fcr9" event={"ID":"050a6565-0515-4f27-8110-51262e515fa2","Type":"ContainerDied","Data":"44f32e02c30bb001e535b9e164041d3f08d03267c615524eb11b06c187bfaca1"}
Oct 30 22:14:16.098417 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:14:16.098390    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-2a25a4-75b5bd6c5-8fcr9"
Oct 30 22:14:16.108892 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:14:16.108863    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/050a6565-0515-4f27-8110-51262e515fa2-kube-api-access-hchcz\" (UniqueName: \"kubernetes.io/projected/050a6565-0515-4f27-8110-51262e515fa2-kube-api-access-hchcz\") pod \"050a6565-0515-4f27-8110-51262e515fa2\" (UID: \"050a6565-0515-4f27-8110-51262e515fa2\") "
Oct 30 22:14:16.109212 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:14:16.109190    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/050a6565-0515-4f27-8110-51262e515fa2-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/050a6565-0515-4f27-8110-51262e515fa2-plugins-config-volume\") pod \"050a6565-0515-4f27-8110-51262e515fa2\" (UID: \"050a6565-0515-4f27-8110-51262e515fa2\") "
Oct 30 22:14:16.109542 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:14:16.109511    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/050a6565-0515-4f27-8110-51262e515fa2-plugins-config-volume" pod "050a6565-0515-4f27-8110-51262e515fa2" (UID: "050a6565-0515-4f27-8110-51262e515fa2"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:14:16.109843 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:14:16.109825    4473 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/050a6565-0515-4f27-8110-51262e515fa2-plugins-config-volume\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:14:16.111204 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:14:16.111183    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/050a6565-0515-4f27-8110-51262e515fa2-kube-api-access-hchcz" pod "050a6565-0515-4f27-8110-51262e515fa2" (UID: "050a6565-0515-4f27-8110-51262e515fa2"). InnerVolumeSpecName "kube-api-access-hchcz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:14:16.211388 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:14:16.211349    4473 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hchcz\" (UniqueName: \"kubernetes.io/projected/050a6565-0515-4f27-8110-51262e515fa2-kube-api-access-hchcz\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:14:17.008042 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:14:17.008009    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-2a25a4-75b5bd6c5-8fcr9"
Oct 30 22:14:17.008492 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:14:17.008097    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-2a25a4-75b5bd6c5-8fcr9" event={"ID":"050a6565-0515-4f27-8110-51262e515fa2","Type":"ContainerDied","Data":"6a980c7da3b5a91a7df25f9498d3a48f908e066c0abf00e6d4da124a1f8fba88"}
Oct 30 22:14:17.008492 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:14:17.008152    4473 scope.go:122] "RemoveContainer" containerID="44f32e02c30bb001e535b9e164041d3f08d03267c615524eb11b06c187bfaca1"
Oct 30 22:14:17.188173 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:14:17.188135    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-2a25a4-75b5bd6c5-8fcr9"]
Oct 30 22:14:17.241321 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:14:17.241283    4473 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-2a25a4-75b5bd6c5-8fcr9"]
Oct 30 22:14:18.360522 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:14:18.360493    4473 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="050a6565-0515-4f27-8110-51262e515fa2" path="/var/lib/kubelet/pods/050a6565-0515-4f27-8110-51262e515fa2/volumes"
Oct 30 22:14:20.340352 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:14:20.340311    4473 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fncvg" containerName="extractor"
Oct 30 22:14:21.005031 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:14:21.005002    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-0a7679-745b8695b5-jlj2t"
Oct 30 22:14:23.983093 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:14:23.983056    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w5-5dff71-7b4c67c84c-xs9qg"
Oct 30 22:14:44.869136 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:14:44.869051    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-8f1d78db-cc4745b9f-w79rb"]
Oct 30 22:14:44.982802 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:14:44.982766    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-8f1d78db-6cf8f64455-rhrhn"]
Oct 30 22:14:49.984153 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:14:49.984097    4473 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-8f1d78db-6cf8f64455-rhrhn" podUID="e8260e3f-690e-40cc-80e7-d31d40f65bc6" containerName="payload-pre-processing" containerID="cri-o://84eef39479c5a7e62f7af7e6e7ac14087ad1dd4aae81713da7fab2b037716e03" gracePeriod=25
Oct 30 22:14:50.271936 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:14:50.271897    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-8f1d78db-6cf8f64455-rhrhn"
Oct 30 22:14:50.326848 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:14:50.326815    4473 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fncvg" containerName="extractor"
Oct 30 22:14:50.360377 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:14:50.360332    4473 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-8f1d78db-cc4745b9f-w79rb" podUID="c4fc1f16-aa8a-4d99-b626-fc6738d32fad" containerName="payload-processing" containerID="cri-o://07c13fd494f71bbd5b64bf9d6bb4b3c44282c11eaf72dbf3e74473b44305f127" gracePeriod=25
Oct 30 22:14:50.400292 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:14:50.400252    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e8260e3f-690e-40cc-80e7-d31d40f65bc6-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e8260e3f-690e-40cc-80e7-d31d40f65bc6-plugins-config-volume\") pod \"e8260e3f-690e-40cc-80e7-d31d40f65bc6\" (UID: \"e8260e3f-690e-40cc-80e7-d31d40f65bc6\") "
Oct 30 22:14:50.400792 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:14:50.400667    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e8260e3f-690e-40cc-80e7-d31d40f65bc6-plugins-config-volume" pod "e8260e3f-690e-40cc-80e7-d31d40f65bc6" (UID: "e8260e3f-690e-40cc-80e7-d31d40f65bc6"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:14:50.401044 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:14:50.401023    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e8260e3f-690e-40cc-80e7-d31d40f65bc6-kube-api-access-mtbkg\" (UniqueName: \"kubernetes.io/projected/e8260e3f-690e-40cc-80e7-d31d40f65bc6-kube-api-access-mtbkg\") pod \"e8260e3f-690e-40cc-80e7-d31d40f65bc6\" (UID: \"e8260e3f-690e-40cc-80e7-d31d40f65bc6\") "
Oct 30 22:14:50.401773 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:14:50.401745    4473 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e8260e3f-690e-40cc-80e7-d31d40f65bc6-plugins-config-volume\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:14:50.403740 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:14:50.403711    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e8260e3f-690e-40cc-80e7-d31d40f65bc6-kube-api-access-mtbkg" pod "e8260e3f-690e-40cc-80e7-d31d40f65bc6" (UID: "e8260e3f-690e-40cc-80e7-d31d40f65bc6"). InnerVolumeSpecName "kube-api-access-mtbkg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:14:50.453441 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:14:50.453409    4473 generic.go:356] "Generic (PLEG): container finished" podID="e8260e3f-690e-40cc-80e7-d31d40f65bc6" containerID="84eef39479c5a7e62f7af7e6e7ac14087ad1dd4aae81713da7fab2b037716e03" exitCode=0
Oct 30 22:14:50.453611 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:14:50.453469    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-8f1d78db-6cf8f64455-rhrhn"
Oct 30 22:14:50.453611 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:14:50.453491    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-8f1d78db-6cf8f64455-rhrhn" event={"ID":"e8260e3f-690e-40cc-80e7-d31d40f65bc6","Type":"ContainerDied","Data":"84eef39479c5a7e62f7af7e6e7ac14087ad1dd4aae81713da7fab2b037716e03"}
Oct 30 22:14:50.453611 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:14:50.453552    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-8f1d78db-6cf8f64455-rhrhn" event={"ID":"e8260e3f-690e-40cc-80e7-d31d40f65bc6","Type":"ContainerDied","Data":"04379eee640b1b686c24fdb3f7d6d790b41ca94256d75c632cf1ccb6ed164ad1"}
Oct 30 22:14:50.453611 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:14:50.453579    4473 scope.go:122] "RemoveContainer" containerID="84eef39479c5a7e62f7af7e6e7ac14087ad1dd4aae81713da7fab2b037716e03"
Oct 30 22:14:50.476205 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:14:50.476178    4473 scope.go:122] "RemoveContainer" containerID="84eef39479c5a7e62f7af7e6e7ac14087ad1dd4aae81713da7fab2b037716e03"
Oct 30 22:14:50.476579 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:14:50.476558    4473 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"84eef39479c5a7e62f7af7e6e7ac14087ad1dd4aae81713da7fab2b037716e03\": container with ID starting with 84eef39479c5a7e62f7af7e6e7ac14087ad1dd4aae81713da7fab2b037716e03 not found: ID does not exist" containerID="84eef39479c5a7e62f7af7e6e7ac14087ad1dd4aae81713da7fab2b037716e03"
Oct 30 22:14:50.476673 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:14:50.476589    4473 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"84eef39479c5a7e62f7af7e6e7ac14087ad1dd4aae81713da7fab2b037716e03"} err="failed to get container status \"84eef39479c5a7e62f7af7e6e7ac14087ad1dd4aae81713da7fab2b037716e03\": rpc error: code = NotFound desc = could not find container \"84eef39479c5a7e62f7af7e6e7ac14087ad1dd4aae81713da7fab2b037716e03\": container with ID starting with 84eef39479c5a7e62f7af7e6e7ac14087ad1dd4aae81713da7fab2b037716e03 not found: ID does not exist"
Oct 30 22:14:50.503106 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:14:50.503021    4473 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mtbkg\" (UniqueName: \"kubernetes.io/projected/e8260e3f-690e-40cc-80e7-d31d40f65bc6-kube-api-access-mtbkg\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:14:50.625743 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:14:50.625707    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-8f1d78db-6cf8f64455-rhrhn"]
Oct 30 22:14:50.626019 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:14:50.626001    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-8f1d78db-cc4745b9f-w79rb"
Oct 30 22:14:50.635161 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:14:50.635133    4473 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-8f1d78db-6cf8f64455-rhrhn"]
Oct 30 22:14:50.705708 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:14:50.705675    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c4fc1f16-aa8a-4d99-b626-fc6738d32fad-kube-api-access-x28zc\" (UniqueName: \"kubernetes.io/projected/c4fc1f16-aa8a-4d99-b626-fc6738d32fad-kube-api-access-x28zc\") pod \"c4fc1f16-aa8a-4d99-b626-fc6738d32fad\" (UID: \"c4fc1f16-aa8a-4d99-b626-fc6738d32fad\") "
Oct 30 22:14:50.705903 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:14:50.705714    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c4fc1f16-aa8a-4d99-b626-fc6738d32fad-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c4fc1f16-aa8a-4d99-b626-fc6738d32fad-plugins-config-volume\") pod \"c4fc1f16-aa8a-4d99-b626-fc6738d32fad\" (UID: \"c4fc1f16-aa8a-4d99-b626-fc6738d32fad\") "
Oct 30 22:14:50.706203 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:14:50.706179    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c4fc1f16-aa8a-4d99-b626-fc6738d32fad-plugins-config-volume" pod "c4fc1f16-aa8a-4d99-b626-fc6738d32fad" (UID: "c4fc1f16-aa8a-4d99-b626-fc6738d32fad"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:14:50.708161 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:14:50.708141    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c4fc1f16-aa8a-4d99-b626-fc6738d32fad-kube-api-access-x28zc" pod "c4fc1f16-aa8a-4d99-b626-fc6738d32fad" (UID: "c4fc1f16-aa8a-4d99-b626-fc6738d32fad"). InnerVolumeSpecName "kube-api-access-x28zc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:14:50.807989 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:14:50.807885    4473 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x28zc\" (UniqueName: \"kubernetes.io/projected/c4fc1f16-aa8a-4d99-b626-fc6738d32fad-kube-api-access-x28zc\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:14:50.807989 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:14:50.807938    4473 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c4fc1f16-aa8a-4d99-b626-fc6738d32fad-plugins-config-volume\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:14:51.462423 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:14:51.462394    4473 generic.go:356] "Generic (PLEG): container finished" podID="c4fc1f16-aa8a-4d99-b626-fc6738d32fad" containerID="07c13fd494f71bbd5b64bf9d6bb4b3c44282c11eaf72dbf3e74473b44305f127" exitCode=0
Oct 30 22:14:51.462882 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:14:51.462441    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-8f1d78db-cc4745b9f-w79rb"
Oct 30 22:14:51.462882 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:14:51.462468    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-8f1d78db-cc4745b9f-w79rb" event={"ID":"c4fc1f16-aa8a-4d99-b626-fc6738d32fad","Type":"ContainerDied","Data":"07c13fd494f71bbd5b64bf9d6bb4b3c44282c11eaf72dbf3e74473b44305f127"}
Oct 30 22:14:51.462882 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:14:51.462514    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-8f1d78db-cc4745b9f-w79rb" event={"ID":"c4fc1f16-aa8a-4d99-b626-fc6738d32fad","Type":"ContainerDied","Data":"e661d3671521d395adf6a3c6005cbf1ad50773324dc2d001e794ebbd9ef9e65b"}
Oct 30 22:14:51.462882 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:14:51.462543    4473 scope.go:122] "RemoveContainer" containerID="07c13fd494f71bbd5b64bf9d6bb4b3c44282c11eaf72dbf3e74473b44305f127"
Oct 30 22:14:51.474312 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:14:51.474283    4473 scope.go:122] "RemoveContainer" containerID="07c13fd494f71bbd5b64bf9d6bb4b3c44282c11eaf72dbf3e74473b44305f127"
Oct 30 22:14:51.474631 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:14:51.474610    4473 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"07c13fd494f71bbd5b64bf9d6bb4b3c44282c11eaf72dbf3e74473b44305f127\": container with ID starting with 07c13fd494f71bbd5b64bf9d6bb4b3c44282c11eaf72dbf3e74473b44305f127 not found: ID does not exist" containerID="07c13fd494f71bbd5b64bf9d6bb4b3c44282c11eaf72dbf3e74473b44305f127"
Oct 30 22:14:51.474766 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:14:51.474650    4473 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"07c13fd494f71bbd5b64bf9d6bb4b3c44282c11eaf72dbf3e74473b44305f127"} err="failed to get container status \"07c13fd494f71bbd5b64bf9d6bb4b3c44282c11eaf72dbf3e74473b44305f127\": rpc error: code = NotFound desc = could not find container \"07c13fd494f71bbd5b64bf9d6bb4b3c44282c11eaf72dbf3e74473b44305f127\": container with ID starting with 07c13fd494f71bbd5b64bf9d6bb4b3c44282c11eaf72dbf3e74473b44305f127 not found: ID does not exist"
Oct 30 22:14:51.630828 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:14:51.630784    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-8f1d78db-cc4745b9f-w79rb"]
Oct 30 22:14:51.634614 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:14:51.634585    4473 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-8f1d78db-cc4745b9f-w79rb"]
Oct 30 22:14:52.369559 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:14:52.369513    4473 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c4fc1f16-aa8a-4d99-b626-fc6738d32fad" path="/var/lib/kubelet/pods/c4fc1f16-aa8a-4d99-b626-fc6738d32fad/volumes"
Oct 30 22:14:52.370081 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:14:52.370066    4473 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e8260e3f-690e-40cc-80e7-d31d40f65bc6" path="/var/lib/kubelet/pods/e8260e3f-690e-40cc-80e7-d31d40f65bc6/volumes"
Oct 30 22:14:56.861956 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:14:56.861892    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-2a25a4-openshift-default-7bfc77bdbf-9prd7"]
Oct 30 22:14:56.862458 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:14:56.862238    4473 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w2-2a25a4-openshift-default-7bfc77bdbf-9prd7" podUID="f0195818-f111-47ca-a386-005ede718851" containerName="istio-proxy" containerID="cri-o://59c46d799b689368b9bcc9c2c5bff2dd3abcdac78f3aebaa014ba1bd1edf5343" gracePeriod=30
Oct 30 22:15:02.155024 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:02.154994    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-2a25a4-openshift-default-7bfc77bdbf-9prd7"
Oct 30 22:15:02.270296 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:02.270257    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f0195818-f111-47ca-a386-005ede718851-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f0195818-f111-47ca-a386-005ede718851-istio-data\") pod \"f0195818-f111-47ca-a386-005ede718851\" (UID: \"f0195818-f111-47ca-a386-005ede718851\") "
Oct 30 22:15:02.270465 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:02.270339    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f0195818-f111-47ca-a386-005ede718851-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0195818-f111-47ca-a386-005ede718851-workload-socket\") pod \"f0195818-f111-47ca-a386-005ede718851\" (UID: \"f0195818-f111-47ca-a386-005ede718851\") "
Oct 30 22:15:02.270465 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:02.270404    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f0195818-f111-47ca-a386-005ede718851-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f0195818-f111-47ca-a386-005ede718851-workload-certs\") pod \"f0195818-f111-47ca-a386-005ede718851\" (UID: \"f0195818-f111-47ca-a386-005ede718851\") "
Oct 30 22:15:02.270614 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:02.270546    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f0195818-f111-47ca-a386-005ede718851-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f0195818-f111-47ca-a386-005ede718851-istio-envoy\") pod \"f0195818-f111-47ca-a386-005ede718851\" (UID: \"f0195818-f111-47ca-a386-005ede718851\") "
Oct 30 22:15:02.270614 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:02.270579    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/f0195818-f111-47ca-a386-005ede718851-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f0195818-f111-47ca-a386-005ede718851-istio-podinfo\") pod \"f0195818-f111-47ca-a386-005ede718851\" (UID: \"f0195818-f111-47ca-a386-005ede718851\") "
Oct 30 22:15:02.270614 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:02.270584    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0195818-f111-47ca-a386-005ede718851-istio-data" pod "f0195818-f111-47ca-a386-005ede718851" (UID: "f0195818-f111-47ca-a386-005ede718851"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:15:02.270784 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:02.270669    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f0195818-f111-47ca-a386-005ede718851-istio-token\" (UniqueName: \"kubernetes.io/projected/f0195818-f111-47ca-a386-005ede718851-istio-token\") pod \"f0195818-f111-47ca-a386-005ede718851\" (UID: \"f0195818-f111-47ca-a386-005ede718851\") "
Oct 30 22:15:02.270784 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:02.270700    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f0195818-f111-47ca-a386-005ede718851-kube-api-access-xlzr6\" (UniqueName: \"kubernetes.io/projected/f0195818-f111-47ca-a386-005ede718851-kube-api-access-xlzr6\") pod \"f0195818-f111-47ca-a386-005ede718851\" (UID: \"f0195818-f111-47ca-a386-005ede718851\") "
Oct 30 22:15:02.270784 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:02.270642    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0195818-f111-47ca-a386-005ede718851-workload-socket" pod "f0195818-f111-47ca-a386-005ede718851" (UID: "f0195818-f111-47ca-a386-005ede718851"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:15:02.270784 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:02.270707    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0195818-f111-47ca-a386-005ede718851-workload-certs" pod "f0195818-f111-47ca-a386-005ede718851" (UID: "f0195818-f111-47ca-a386-005ede718851"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:15:02.270784 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:02.270776    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f0195818-f111-47ca-a386-005ede718851-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f0195818-f111-47ca-a386-005ede718851-istiod-ca-cert\") pod \"f0195818-f111-47ca-a386-005ede718851\" (UID: \"f0195818-f111-47ca-a386-005ede718851\") "
Oct 30 22:15:02.271080 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:02.270875    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f0195818-f111-47ca-a386-005ede718851-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0195818-f111-47ca-a386-005ede718851-credential-socket\") pod \"f0195818-f111-47ca-a386-005ede718851\" (UID: \"f0195818-f111-47ca-a386-005ede718851\") "
Oct 30 22:15:02.271284 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:02.271253    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f0195818-f111-47ca-a386-005ede718851-istiod-ca-cert" pod "f0195818-f111-47ca-a386-005ede718851" (UID: "f0195818-f111-47ca-a386-005ede718851"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:15:02.271499 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:02.271277    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0195818-f111-47ca-a386-005ede718851-credential-socket" pod "f0195818-f111-47ca-a386-005ede718851" (UID: "f0195818-f111-47ca-a386-005ede718851"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:15:02.271623 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:02.271562    4473 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0195818-f111-47ca-a386-005ede718851-workload-socket\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:15:02.271623 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:02.271584    4473 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f0195818-f111-47ca-a386-005ede718851-workload-certs\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:15:02.271623 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:02.271598    4473 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f0195818-f111-47ca-a386-005ede718851-istiod-ca-cert\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:15:02.271623 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:02.271612    4473 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0195818-f111-47ca-a386-005ede718851-credential-socket\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:15:02.271815 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:02.271625    4473 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f0195818-f111-47ca-a386-005ede718851-istio-data\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:15:02.273273 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:02.273246    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f0195818-f111-47ca-a386-005ede718851-istio-podinfo" pod "f0195818-f111-47ca-a386-005ede718851" (UID: "f0195818-f111-47ca-a386-005ede718851"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:15:02.273396 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:02.273375    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f0195818-f111-47ca-a386-005ede718851-kube-api-access-xlzr6" pod "f0195818-f111-47ca-a386-005ede718851" (UID: "f0195818-f111-47ca-a386-005ede718851"). InnerVolumeSpecName "kube-api-access-xlzr6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:15:02.273583 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:02.273562    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0195818-f111-47ca-a386-005ede718851-istio-envoy" pod "f0195818-f111-47ca-a386-005ede718851" (UID: "f0195818-f111-47ca-a386-005ede718851"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:15:02.273652 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:02.273597    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f0195818-f111-47ca-a386-005ede718851-istio-token" pod "f0195818-f111-47ca-a386-005ede718851" (UID: "f0195818-f111-47ca-a386-005ede718851"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:15:02.373473 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:02.373443    4473 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f0195818-f111-47ca-a386-005ede718851-istio-envoy\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:15:02.373473 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:02.373473    4473 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f0195818-f111-47ca-a386-005ede718851-istio-podinfo\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:15:02.373658 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:02.373486    4473 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f0195818-f111-47ca-a386-005ede718851-istio-token\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:15:02.373658 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:02.373522    4473 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xlzr6\" (UniqueName: \"kubernetes.io/projected/f0195818-f111-47ca-a386-005ede718851-kube-api-access-xlzr6\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:15:02.576423 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:02.576324    4473 generic.go:356] "Generic (PLEG): container finished" podID="f0195818-f111-47ca-a386-005ede718851" containerID="59c46d799b689368b9bcc9c2c5bff2dd3abcdac78f3aebaa014ba1bd1edf5343" exitCode=0
Oct 30 22:15:02.576423 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:02.576381    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-2a25a4-openshift-default-7bfc77bdbf-9prd7"
Oct 30 22:15:02.576650 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:02.576384    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-2a25a4-openshift-default-7bfc77bdbf-9prd7" event={"ID":"f0195818-f111-47ca-a386-005ede718851","Type":"ContainerDied","Data":"59c46d799b689368b9bcc9c2c5bff2dd3abcdac78f3aebaa014ba1bd1edf5343"}
Oct 30 22:15:02.576650 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:02.576527    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-2a25a4-openshift-default-7bfc77bdbf-9prd7" event={"ID":"f0195818-f111-47ca-a386-005ede718851","Type":"ContainerDied","Data":"08e3a27608aea71044a60e510966e054159261031c7896f8a552cb01906f06ca"}
Oct 30 22:15:02.576650 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:02.576560    4473 scope.go:122] "RemoveContainer" containerID="59c46d799b689368b9bcc9c2c5bff2dd3abcdac78f3aebaa014ba1bd1edf5343"
Oct 30 22:15:02.590807 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:02.590782    4473 scope.go:122] "RemoveContainer" containerID="4d00e9f44c1f24b9b0a4c0d1666f42731d1a25290cdb7c1424a51317a0f45cc2"
Oct 30 22:15:02.601859 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:02.601834    4473 scope.go:122] "RemoveContainer" containerID="59c46d799b689368b9bcc9c2c5bff2dd3abcdac78f3aebaa014ba1bd1edf5343"
Oct 30 22:15:02.602242 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:15:02.602216    4473 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"59c46d799b689368b9bcc9c2c5bff2dd3abcdac78f3aebaa014ba1bd1edf5343\": container with ID starting with 59c46d799b689368b9bcc9c2c5bff2dd3abcdac78f3aebaa014ba1bd1edf5343 not found: ID does not exist" containerID="59c46d799b689368b9bcc9c2c5bff2dd3abcdac78f3aebaa014ba1bd1edf5343"
Oct 30 22:15:02.602335 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:02.602256    4473 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"59c46d799b689368b9bcc9c2c5bff2dd3abcdac78f3aebaa014ba1bd1edf5343"} err="failed to get container status \"59c46d799b689368b9bcc9c2c5bff2dd3abcdac78f3aebaa014ba1bd1edf5343\": rpc error: code = NotFound desc = could not find container \"59c46d799b689368b9bcc9c2c5bff2dd3abcdac78f3aebaa014ba1bd1edf5343\": container with ID starting with 59c46d799b689368b9bcc9c2c5bff2dd3abcdac78f3aebaa014ba1bd1edf5343 not found: ID does not exist"
Oct 30 22:15:02.602335 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:02.602283    4473 scope.go:122] "RemoveContainer" containerID="4d00e9f44c1f24b9b0a4c0d1666f42731d1a25290cdb7c1424a51317a0f45cc2"
Oct 30 22:15:02.602606 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:15:02.602585    4473 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4d00e9f44c1f24b9b0a4c0d1666f42731d1a25290cdb7c1424a51317a0f45cc2\": container with ID starting with 4d00e9f44c1f24b9b0a4c0d1666f42731d1a25290cdb7c1424a51317a0f45cc2 not found: ID does not exist" containerID="4d00e9f44c1f24b9b0a4c0d1666f42731d1a25290cdb7c1424a51317a0f45cc2"
Oct 30 22:15:02.602674 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:02.602615    4473 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4d00e9f44c1f24b9b0a4c0d1666f42731d1a25290cdb7c1424a51317a0f45cc2"} err="failed to get container status \"4d00e9f44c1f24b9b0a4c0d1666f42731d1a25290cdb7c1424a51317a0f45cc2\": rpc error: code = NotFound desc = could not find container \"4d00e9f44c1f24b9b0a4c0d1666f42731d1a25290cdb7c1424a51317a0f45cc2\": container with ID starting with 4d00e9f44c1f24b9b0a4c0d1666f42731d1a25290cdb7c1424a51317a0f45cc2 not found: ID does not exist"
Oct 30 22:15:02.746525 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:02.746487    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-2a25a4-openshift-default-7bfc77bdbf-9prd7"]
Oct 30 22:15:02.797773 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:02.797729    4473 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-2a25a4-openshift-default-7bfc77bdbf-9prd7"]
Oct 30 22:15:04.360280 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:04.360237    4473 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f0195818-f111-47ca-a386-005ede718851" path="/var/lib/kubelet/pods/f0195818-f111-47ca-a386-005ede718851/volumes"
Oct 30 22:15:06.611525 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:06.611493    4473 generic.go:356] "Generic (PLEG): container finished" podID="b7789a8c-e252-4a6e-b026-3b5ce2b39b41" containerID="29565cd3fb267390ae283bf15cf0534a08cb0883922fd44fc5f909778f42b88b" exitCode=0
Oct 30 22:15:06.612040 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:06.611572    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w6-0a7679-openshift-default-5b6c6cc645-p8j4w" event={"ID":"b7789a8c-e252-4a6e-b026-3b5ce2b39b41","Type":"ContainerDied","Data":"29565cd3fb267390ae283bf15cf0534a08cb0883922fd44fc5f909778f42b88b"}
Oct 30 22:15:06.612092 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:06.612058    4473 scope.go:122] "RemoveContainer" containerID="29565cd3fb267390ae283bf15cf0534a08cb0883922fd44fc5f909778f42b88b"
Oct 30 22:15:06.613798 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:06.613763    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:15:06.613965 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:06.613851    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:15:06.613965 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:06.613899    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:15:06.720700 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:06.720651    4473 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7314-6b259a"]
Oct 30 22:15:06.721621 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:06.721588    4473 cpu_manager.go:414] "containerMap: removing container" podUID="050a6565-0515-4f27-8110-51262e515fa2" containerName="payload-processing"
Oct 30 22:15:06.721772 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:06.721612    4473 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="050a6565-0515-4f27-8110-51262e515fa2" containerName="payload-processing"
Oct 30 22:15:06.721838 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:06.721791    4473 cpu_manager.go:414] "containerMap: removing container" podUID="e8260e3f-690e-40cc-80e7-d31d40f65bc6" containerName="payload-pre-processing"
Oct 30 22:15:06.721838 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:06.721802    4473 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e8260e3f-690e-40cc-80e7-d31d40f65bc6" containerName="payload-pre-processing"
Oct 30 22:15:06.721972 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:06.721840    4473 cpu_manager.go:414] "containerMap: removing container" podUID="f0195818-f111-47ca-a386-005ede718851" containerName="istio-proxy"
Oct 30 22:15:06.721972 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:06.721848    4473 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f0195818-f111-47ca-a386-005ede718851" containerName="istio-proxy"
Oct 30 22:15:06.721972 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:06.721870    4473 cpu_manager.go:414] "containerMap: removing container" podUID="c4fc1f16-aa8a-4d99-b626-fc6738d32fad" containerName="payload-processing"
Oct 30 22:15:06.721972 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:06.721877    4473 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c4fc1f16-aa8a-4d99-b626-fc6738d32fad" containerName="payload-processing"
Oct 30 22:15:06.721972 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:06.721894    4473 cpu_manager.go:414] "containerMap: removing container" podUID="f0195818-f111-47ca-a386-005ede718851" containerName="istio-proxy"
Oct 30 22:15:06.721972 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:06.721902    4473 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f0195818-f111-47ca-a386-005ede718851" containerName="istio-proxy"
Oct 30 22:15:06.722670 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:06.722537    4473 memory_manager.go:365] "RemoveStaleState removing state" podUID="f0195818-f111-47ca-a386-005ede718851" containerName="istio-proxy"
Oct 30 22:15:06.722670 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:06.722565    4473 memory_manager.go:365] "RemoveStaleState removing state" podUID="c4fc1f16-aa8a-4d99-b626-fc6738d32fad" containerName="payload-processing"
Oct 30 22:15:06.722670 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:06.722579    4473 memory_manager.go:365] "RemoveStaleState removing state" podUID="f0195818-f111-47ca-a386-005ede718851" containerName="istio-proxy"
Oct 30 22:15:06.722670 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:06.722593    4473 memory_manager.go:365] "RemoveStaleState removing state" podUID="050a6565-0515-4f27-8110-51262e515fa2" containerName="payload-processing"
Oct 30 22:15:06.722670 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:06.722614    4473 memory_manager.go:365] "RemoveStaleState removing state" podUID="e8260e3f-690e-40cc-80e7-d31d40f65bc6" containerName="payload-pre-processing"
Oct 30 22:15:06.727959 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:06.727118    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7314-6b259a"
Oct 30 22:15:06.728301 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:06.728280    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7314-6b259a"]
Oct 30 22:15:06.781542 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:06.781423    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-qmjfd\""
Oct 30 22:15:06.851409 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:06.851369    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rzk2j\" (UniqueName: \"kubernetes.io/projected/dc340a9f-0368-4d8f-9047-be78b84f8a4c-kube-api-access-rzk2j\") pod \"test-curl-7314-6b259a\" (UID: \"dc340a9f-0368-4d8f-9047-be78b84f8a4c\") " pod="opendatahub/test-curl-7314-6b259a"
Oct 30 22:15:06.953766 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:06.953710    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rzk2j\" (UniqueName: \"kubernetes.io/projected/dc340a9f-0368-4d8f-9047-be78b84f8a4c-kube-api-access-rzk2j\") pod \"test-curl-7314-6b259a\" (UID: \"dc340a9f-0368-4d8f-9047-be78b84f8a4c\") " pod="opendatahub/test-curl-7314-6b259a"
Oct 30 22:15:07.012063 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:07.012028    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rzk2j\" (UniqueName: \"kubernetes.io/projected/dc340a9f-0368-4d8f-9047-be78b84f8a4c-kube-api-access-rzk2j\") pod \"test-curl-7314-6b259a\" (UID: \"dc340a9f-0368-4d8f-9047-be78b84f8a4c\") " pod="opendatahub/test-curl-7314-6b259a"
Oct 30 22:15:07.049006 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:07.048962    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7314-6b259a"
Oct 30 22:15:07.458895 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:07.458571    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7314-6b259a"]
Oct 30 22:15:07.621176 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:07.621136    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w6-0a7679-openshift-default-5b6c6cc645-p8j4w" event={"ID":"b7789a8c-e252-4a6e-b026-3b5ce2b39b41","Type":"ContainerStarted","Data":"bbaffd0baa49a1a171003426b82cc8248e39da93ec95fef1ce4c8d1d79e74e2c"}
Oct 30 22:15:07.622598 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:07.622569    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7314-6b259a" event={"ID":"dc340a9f-0368-4d8f-9047-be78b84f8a4c","Type":"ContainerStarted","Data":"7bd79e153d73e015153860e2d34e0f9e8da1c2a3339c5aeedb66c33d374be5d2"}
Oct 30 22:15:07.865165 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:07.861913    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w6-0a7679-openshift-default-5b6c6cc645-p8j4w"
Oct 30 22:15:07.865165 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:07.862122    4473 patch_prober.go:29] interesting pod/e2e-shared-b-w6-0a7679-openshift-default-5b6c6cc645-p8j4w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.131:15021/healthz/ready\": dial tcp 10.133.0.131:15021: connect: connection refused" start-of-body=
Oct 30 22:15:07.865165 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:07.862192    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w6-0a7679-openshift-default-5b6c6cc645-p8j4w" podUID="b7789a8c-e252-4a6e-b026-3b5ce2b39b41" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.131:15021/healthz/ready\": dial tcp 10.133.0.131:15021: connect: connection refused"
Oct 30 22:15:08.862409 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:08.862363    4473 patch_prober.go:29] interesting pod/e2e-shared-b-w6-0a7679-openshift-default-5b6c6cc645-p8j4w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.131:15021/healthz/ready\": dial tcp 10.133.0.131:15021: connect: connection refused" start-of-body=
Oct 30 22:15:08.862968 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:08.862435    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w6-0a7679-openshift-default-5b6c6cc645-p8j4w" podUID="b7789a8c-e252-4a6e-b026-3b5ce2b39b41" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.131:15021/healthz/ready\": dial tcp 10.133.0.131:15021: connect: connection refused"
Oct 30 22:15:09.351746 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:09.351711    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-7c826c-9b5d559f7-dr9ng"]
Oct 30 22:15:09.465287 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:09.463003    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-7c826c-86f4595c8c-llmxr"]
Oct 30 22:15:09.665774 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:09.665735    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7314-6b259a" event={"ID":"dc340a9f-0368-4d8f-9047-be78b84f8a4c","Type":"ContainerStarted","Data":"0aa26b41c1e20ea794e436449f600fe36cbba080cd9649c9a86230e63858242e"}
Oct 30 22:15:09.912956 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:09.908950    4473 patch_prober.go:29] interesting pod/e2e-shared-b-w6-0a7679-openshift-default-5b6c6cc645-p8j4w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.131:15021/healthz/ready\": dial tcp 10.133.0.131:15021: connect: connection refused" start-of-body=
Oct 30 22:15:09.912956 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:09.909007    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w6-0a7679-openshift-default-5b6c6cc645-p8j4w" podUID="b7789a8c-e252-4a6e-b026-3b5ce2b39b41" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.131:15021/healthz/ready\": dial tcp 10.133.0.131:15021: connect: connection refused"
Oct 30 22:15:09.919970 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:09.919056    4473 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/test-curl-7314-6b259a" podStartSLOduration=1.878042488 podStartE2EDuration="3.919032224s" totalImagesPullingTime="2.040989736s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:15:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:15:09.88597093 +0000 UTC m=+1955.659339416" watchObservedRunningTime="2026-10-30 22:15:09.919032224 +0000 UTC m=+1955.692400718"
Oct 30 22:15:10.378103 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:10.378070    4473 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-db492e-676764d957-6j782"]
Oct 30 22:15:10.382976 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:10.382910    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-db492e-676764d957-6j782"
Oct 30 22:15:10.416352 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:10.416300    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8fa4be84-31c4-4076-b5cd-d72fcbbe0e5d-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w0-db492e-676764d957-6j782\" (UID: \"8fa4be84-31c4-4076-b5cd-d72fcbbe0e5d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-db492e-676764d957-6j782"
Oct 30 22:15:10.416521 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:10.416458    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8fa4be84-31c4-4076-b5cd-d72fcbbe0e5d-maas-api-tls\") pod \"maas-api-e2e-shared-b-w0-db492e-676764d957-6j782\" (UID: \"8fa4be84-31c4-4076-b5cd-d72fcbbe0e5d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-db492e-676764d957-6j782"
Oct 30 22:15:10.416675 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:10.416650    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9hhpk\" (UniqueName: \"kubernetes.io/projected/8fa4be84-31c4-4076-b5cd-d72fcbbe0e5d-kube-api-access-9hhpk\") pod \"maas-api-e2e-shared-b-w0-db492e-676764d957-6j782\" (UID: \"8fa4be84-31c4-4076-b5cd-d72fcbbe0e5d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-db492e-676764d957-6j782"
Oct 30 22:15:10.417030 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:10.417008    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8fa4be84-31c4-4076-b5cd-d72fcbbe0e5d-metrics-certs\") pod \"maas-api-e2e-shared-b-w0-db492e-676764d957-6j782\" (UID: \"8fa4be84-31c4-4076-b5cd-d72fcbbe0e5d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-db492e-676764d957-6j782"
Oct 30 22:15:10.447530 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:10.447492    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w0-db492e-serving-cert\""
Oct 30 22:15:10.458367 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:10.458333    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-db492e-676764d957-6j782"]
Oct 30 22:15:10.518270 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:10.518233    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8fa4be84-31c4-4076-b5cd-d72fcbbe0e5d-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w0-db492e-676764d957-6j782\" (UID: \"8fa4be84-31c4-4076-b5cd-d72fcbbe0e5d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-db492e-676764d957-6j782"
Oct 30 22:15:10.518448 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:10.518366    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8fa4be84-31c4-4076-b5cd-d72fcbbe0e5d-maas-api-tls\") pod \"maas-api-e2e-shared-b-w0-db492e-676764d957-6j782\" (UID: \"8fa4be84-31c4-4076-b5cd-d72fcbbe0e5d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-db492e-676764d957-6j782"
Oct 30 22:15:10.518506 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:10.518476    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9hhpk\" (UniqueName: \"kubernetes.io/projected/8fa4be84-31c4-4076-b5cd-d72fcbbe0e5d-kube-api-access-9hhpk\") pod \"maas-api-e2e-shared-b-w0-db492e-676764d957-6j782\" (UID: \"8fa4be84-31c4-4076-b5cd-d72fcbbe0e5d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-db492e-676764d957-6j782"
Oct 30 22:15:10.518819 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:10.518559    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8fa4be84-31c4-4076-b5cd-d72fcbbe0e5d-metrics-certs\") pod \"maas-api-e2e-shared-b-w0-db492e-676764d957-6j782\" (UID: \"8fa4be84-31c4-4076-b5cd-d72fcbbe0e5d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-db492e-676764d957-6j782"
Oct 30 22:15:10.518965 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:10.518835    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8fa4be84-31c4-4076-b5cd-d72fcbbe0e5d-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w0-db492e-676764d957-6j782\" (UID: \"8fa4be84-31c4-4076-b5cd-d72fcbbe0e5d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-db492e-676764d957-6j782"
Oct 30 22:15:10.519115 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:15:10.519097    4473 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-db492e-serving-cert: secret "maas-api-e2e-shared-b-w0-db492e-serving-cert" not found
Oct 30 22:15:10.519237 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:15:10.519226    4473 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8fa4be84-31c4-4076-b5cd-d72fcbbe0e5d-maas-api-tls podName:8fa4be84-31c4-4076-b5cd-d72fcbbe0e5d nodeName:}" failed. No retries permitted until 2026-10-30 22:15:11.019208781 +0000 UTC m=+1956.792577261 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/8fa4be84-31c4-4076-b5cd-d72fcbbe0e5d-maas-api-tls") pod "maas-api-e2e-shared-b-w0-db492e-676764d957-6j782" (UID: "8fa4be84-31c4-4076-b5cd-d72fcbbe0e5d") : secret "maas-api-e2e-shared-b-w0-db492e-serving-cert" not found
Oct 30 22:15:10.522532 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:10.522503    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8fa4be84-31c4-4076-b5cd-d72fcbbe0e5d-metrics-certs\") pod \"maas-api-e2e-shared-b-w0-db492e-676764d957-6j782\" (UID: \"8fa4be84-31c4-4076-b5cd-d72fcbbe0e5d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-db492e-676764d957-6j782"
Oct 30 22:15:10.576157 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:10.576124    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9hhpk\" (UniqueName: \"kubernetes.io/projected/8fa4be84-31c4-4076-b5cd-d72fcbbe0e5d-kube-api-access-9hhpk\") pod \"maas-api-e2e-shared-b-w0-db492e-676764d957-6j782\" (UID: \"8fa4be84-31c4-4076-b5cd-d72fcbbe0e5d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-db492e-676764d957-6j782"
Oct 30 22:15:10.919958 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:10.910800    4473 patch_prober.go:29] interesting pod/e2e-shared-b-w6-0a7679-openshift-default-5b6c6cc645-p8j4w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.131:15021/healthz/ready\": dial tcp 10.133.0.131:15021: connect: connection refused" start-of-body=
Oct 30 22:15:10.920577 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:10.920539    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w6-0a7679-openshift-default-5b6c6cc645-p8j4w" podUID="b7789a8c-e252-4a6e-b026-3b5ce2b39b41" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.131:15021/healthz/ready\": dial tcp 10.133.0.131:15021: connect: connection refused"
Oct 30 22:15:11.093952 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:11.091108    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8fa4be84-31c4-4076-b5cd-d72fcbbe0e5d-maas-api-tls\") pod \"maas-api-e2e-shared-b-w0-db492e-676764d957-6j782\" (UID: \"8fa4be84-31c4-4076-b5cd-d72fcbbe0e5d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-db492e-676764d957-6j782"
Oct 30 22:15:11.258864 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:11.258772    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8fa4be84-31c4-4076-b5cd-d72fcbbe0e5d-maas-api-tls\") pod \"maas-api-e2e-shared-b-w0-db492e-676764d957-6j782\" (UID: \"8fa4be84-31c4-4076-b5cd-d72fcbbe0e5d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-db492e-676764d957-6j782"
Oct 30 22:15:11.302946 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:11.302343    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-db492e-676764d957-6j782"
Oct 30 22:15:11.779007 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:11.778181    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7314-6b259a"]
Oct 30 22:15:11.779007 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:11.778372    4473 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/test-curl-7314-6b259a" podUID="dc340a9f-0368-4d8f-9047-be78b84f8a4c" containerName="test-curl-7314-6b259a" containerID="cri-o://0aa26b41c1e20ea794e436449f600fe36cbba080cd9649c9a86230e63858242e" gracePeriod=2
Oct 30 22:15:11.782784 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:11.782753    4473 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7314-6b259a"]
Oct 30 22:15:11.851991 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:15:11.848610    4473 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7314-6b259a\" is forbidden: User \"system:node:ip-10-0-140-18.us-west-1.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-140-18.us-west-1.compute.internal' and this object" podUID="dc340a9f-0368-4d8f-9047-be78b84f8a4c" pod="opendatahub/test-curl-7314-6b259a"
Oct 30 22:15:11.862979 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:11.862613    4473 patch_prober.go:29] interesting pod/e2e-shared-b-w6-0a7679-openshift-default-5b6c6cc645-p8j4w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.131:15021/healthz/ready\": dial tcp 10.133.0.131:15021: connect: connection refused" start-of-body=
Oct 30 22:15:11.862979 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:11.862665    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w6-0a7679-openshift-default-5b6c6cc645-p8j4w" podUID="b7789a8c-e252-4a6e-b026-3b5ce2b39b41" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.131:15021/healthz/ready\": dial tcp 10.133.0.131:15021: connect: connection refused"
Oct 30 22:15:11.962979 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:11.962741    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-db492e-676764d957-6j782"]
Oct 30 22:15:12.692906 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:12.692873    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-db492e-676764d957-6j782" event={"ID":"8fa4be84-31c4-4076-b5cd-d72fcbbe0e5d","Type":"ContainerStarted","Data":"667d07ae516f88876063ebfef0a72156a27b102f33f0350cbe7b4dced2822e04"}
Oct 30 22:15:12.862397 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:12.862361    4473 patch_prober.go:29] interesting pod/e2e-shared-b-w6-0a7679-openshift-default-5b6c6cc645-p8j4w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.131:15021/healthz/ready\": dial tcp 10.133.0.131:15021: connect: connection refused" start-of-body=
Oct 30 22:15:12.862598 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:12.862413    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w6-0a7679-openshift-default-5b6c6cc645-p8j4w" podUID="b7789a8c-e252-4a6e-b026-3b5ce2b39b41" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.131:15021/healthz/ready\": dial tcp 10.133.0.131:15021: connect: connection refused"
Oct 30 22:15:13.701989 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:13.701911    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-db492e-676764d957-6j782" event={"ID":"8fa4be84-31c4-4076-b5cd-d72fcbbe0e5d","Type":"ContainerStarted","Data":"bab83643a56b51bb9fcc7dd530793c7841bf7669d1e5d2ae189fee4fbc5e99d6"}
Oct 30 22:15:13.702495 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:13.702022    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-db492e-676764d957-6j782"
Oct 30 22:15:13.859187 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:13.859104    4473 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-db492e-676764d957-6j782" podStartSLOduration=2.985705403 podStartE2EDuration="3.859079688s" totalImagesPullingTime="873.374285ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:15:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:15:13.853365789 +0000 UTC m=+1959.626734276" watchObservedRunningTime="2026-10-30 22:15:13.859079688 +0000 UTC m=+1959.632448175"
Oct 30 22:15:13.915492 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:13.910178    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w6-0a7679-openshift-default-5b6c6cc645-p8j4w"
Oct 30 22:15:13.915492 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:13.912518    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w6-0a7679-openshift-default-5b6c6cc645-p8j4w"
Oct 30 22:15:13.915492 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:13.915286    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w6-0a7679-openshift-default-5b6c6cc645-p8j4w"
Oct 30 22:15:14.160553 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:14.160515    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7314-6b259a"
Oct 30 22:15:14.224961 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:14.224931    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/dc340a9f-0368-4d8f-9047-be78b84f8a4c-kube-api-access-rzk2j\" (UniqueName: \"kubernetes.io/projected/dc340a9f-0368-4d8f-9047-be78b84f8a4c-kube-api-access-rzk2j\") pod \"dc340a9f-0368-4d8f-9047-be78b84f8a4c\" (UID: \"dc340a9f-0368-4d8f-9047-be78b84f8a4c\") "
Oct 30 22:15:14.227528 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:14.227500    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dc340a9f-0368-4d8f-9047-be78b84f8a4c-kube-api-access-rzk2j" pod "dc340a9f-0368-4d8f-9047-be78b84f8a4c" (UID: "dc340a9f-0368-4d8f-9047-be78b84f8a4c"). InnerVolumeSpecName "kube-api-access-rzk2j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:15:14.327367 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:14.327275    4473 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rzk2j\" (UniqueName: \"kubernetes.io/projected/dc340a9f-0368-4d8f-9047-be78b84f8a4c-kube-api-access-rzk2j\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:15:14.352202 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:14.352157    4473 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-7c826c-9b5d559f7-dr9ng" podUID="3e170928-4f15-43bf-9ad9-1a38b9bb0950" containerName="payload-processing" containerID="cri-o://4b9aceab0c6cef1020b7515554a5bde11aac311ea6253b7831704ed7e275d6ff" gracePeriod=25
Oct 30 22:15:14.373397 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:14.373360    4473 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="dc340a9f-0368-4d8f-9047-be78b84f8a4c" path="/var/lib/kubelet/pods/dc340a9f-0368-4d8f-9047-be78b84f8a4c/volumes"
Oct 30 22:15:14.464360 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:14.464300    4473 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-7c826c-86f4595c8c-llmxr" podUID="1ff2c276-f027-4581-94b1-31ce87f60d59" containerName="payload-pre-processing" containerID="cri-o://dc0af6b693a9399be3a1857b89f03a4f1090502ae70fc9a7bf64716c2017fec5" gracePeriod=25
Oct 30 22:15:14.621449 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:14.621420    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-7c826c-9b5d559f7-dr9ng"
Oct 30 22:15:14.714103 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:14.714060    4473 generic.go:356] "Generic (PLEG): container finished" podID="3e170928-4f15-43bf-9ad9-1a38b9bb0950" containerID="4b9aceab0c6cef1020b7515554a5bde11aac311ea6253b7831704ed7e275d6ff" exitCode=0
Oct 30 22:15:14.714494 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:14.714238    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-7c826c-9b5d559f7-dr9ng" event={"ID":"3e170928-4f15-43bf-9ad9-1a38b9bb0950","Type":"ContainerDied","Data":"4b9aceab0c6cef1020b7515554a5bde11aac311ea6253b7831704ed7e275d6ff"}
Oct 30 22:15:14.714494 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:14.714295    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-7c826c-9b5d559f7-dr9ng" event={"ID":"3e170928-4f15-43bf-9ad9-1a38b9bb0950","Type":"ContainerDied","Data":"aeb86244822c1fb7166d69929e5bcd5b304f2b8eb15198c5073687e41494d082"}
Oct 30 22:15:14.714494 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:14.714324    4473 scope.go:122] "RemoveContainer" containerID="4b9aceab0c6cef1020b7515554a5bde11aac311ea6253b7831704ed7e275d6ff"
Oct 30 22:15:14.714494 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:14.714485    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-7c826c-9b5d559f7-dr9ng"
Oct 30 22:15:14.716512 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:14.716487    4473 generic.go:356] "Generic (PLEG): container finished" podID="dc340a9f-0368-4d8f-9047-be78b84f8a4c" containerID="0aa26b41c1e20ea794e436449f600fe36cbba080cd9649c9a86230e63858242e" exitCode=137
Oct 30 22:15:14.716671 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:14.716656    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7314-6b259a"
Oct 30 22:15:14.720115 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:14.720092    4473 generic.go:356] "Generic (PLEG): container finished" podID="1ff2c276-f027-4581-94b1-31ce87f60d59" containerID="dc0af6b693a9399be3a1857b89f03a4f1090502ae70fc9a7bf64716c2017fec5" exitCode=0
Oct 30 22:15:14.720252 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:14.720156    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-7c826c-86f4595c8c-llmxr" event={"ID":"1ff2c276-f027-4581-94b1-31ce87f60d59","Type":"ContainerDied","Data":"dc0af6b693a9399be3a1857b89f03a4f1090502ae70fc9a7bf64716c2017fec5"}
Oct 30 22:15:14.728109 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:14.728071    4473 scope.go:122] "RemoveContainer" containerID="4b9aceab0c6cef1020b7515554a5bde11aac311ea6253b7831704ed7e275d6ff"
Oct 30 22:15:14.728593 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:15:14.728454    4473 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4b9aceab0c6cef1020b7515554a5bde11aac311ea6253b7831704ed7e275d6ff\": container with ID starting with 4b9aceab0c6cef1020b7515554a5bde11aac311ea6253b7831704ed7e275d6ff not found: ID does not exist" containerID="4b9aceab0c6cef1020b7515554a5bde11aac311ea6253b7831704ed7e275d6ff"
Oct 30 22:15:14.728593 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:14.728497    4473 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4b9aceab0c6cef1020b7515554a5bde11aac311ea6253b7831704ed7e275d6ff"} err="failed to get container status \"4b9aceab0c6cef1020b7515554a5bde11aac311ea6253b7831704ed7e275d6ff\": rpc error: code = NotFound desc = could not find container \"4b9aceab0c6cef1020b7515554a5bde11aac311ea6253b7831704ed7e275d6ff\": container with ID starting with 4b9aceab0c6cef1020b7515554a5bde11aac311ea6253b7831704ed7e275d6ff not found: ID does not exist"
Oct 30 22:15:14.728593 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:14.728516    4473 scope.go:122] "RemoveContainer" containerID="0aa26b41c1e20ea794e436449f600fe36cbba080cd9649c9a86230e63858242e"
Oct 30 22:15:14.734656 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:14.733444    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3e170928-4f15-43bf-9ad9-1a38b9bb0950-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3e170928-4f15-43bf-9ad9-1a38b9bb0950-plugins-config-volume\") pod \"3e170928-4f15-43bf-9ad9-1a38b9bb0950\" (UID: \"3e170928-4f15-43bf-9ad9-1a38b9bb0950\") "
Oct 30 22:15:14.734656 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:14.733802    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3e170928-4f15-43bf-9ad9-1a38b9bb0950-kube-api-access-dhvz8\" (UniqueName: \"kubernetes.io/projected/3e170928-4f15-43bf-9ad9-1a38b9bb0950-kube-api-access-dhvz8\") pod \"3e170928-4f15-43bf-9ad9-1a38b9bb0950\" (UID: \"3e170928-4f15-43bf-9ad9-1a38b9bb0950\") "
Oct 30 22:15:14.734656 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:14.733873    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3e170928-4f15-43bf-9ad9-1a38b9bb0950-plugins-config-volume" pod "3e170928-4f15-43bf-9ad9-1a38b9bb0950" (UID: "3e170928-4f15-43bf-9ad9-1a38b9bb0950"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:15:14.734656 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:14.734536    4473 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3e170928-4f15-43bf-9ad9-1a38b9bb0950-plugins-config-volume\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:15:14.736434 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:14.736410    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3e170928-4f15-43bf-9ad9-1a38b9bb0950-kube-api-access-dhvz8" pod "3e170928-4f15-43bf-9ad9-1a38b9bb0950" (UID: "3e170928-4f15-43bf-9ad9-1a38b9bb0950"). InnerVolumeSpecName "kube-api-access-dhvz8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:15:14.739906 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:14.739879    4473 scope.go:122] "RemoveContainer" containerID="0aa26b41c1e20ea794e436449f600fe36cbba080cd9649c9a86230e63858242e"
Oct 30 22:15:14.740275 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:15:14.740248    4473 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0aa26b41c1e20ea794e436449f600fe36cbba080cd9649c9a86230e63858242e\": container with ID starting with 0aa26b41c1e20ea794e436449f600fe36cbba080cd9649c9a86230e63858242e not found: ID does not exist" containerID="0aa26b41c1e20ea794e436449f600fe36cbba080cd9649c9a86230e63858242e"
Oct 30 22:15:14.740360 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:14.740285    4473 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0aa26b41c1e20ea794e436449f600fe36cbba080cd9649c9a86230e63858242e"} err="failed to get container status \"0aa26b41c1e20ea794e436449f600fe36cbba080cd9649c9a86230e63858242e\": rpc error: code = NotFound desc = could not find container \"0aa26b41c1e20ea794e436449f600fe36cbba080cd9649c9a86230e63858242e\": container with ID starting with 0aa26b41c1e20ea794e436449f600fe36cbba080cd9649c9a86230e63858242e not found: ID does not exist"
Oct 30 22:15:14.754390 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:14.754357    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-7c826c-86f4595c8c-llmxr"
Oct 30 22:15:14.836729 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:14.836645    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1ff2c276-f027-4581-94b1-31ce87f60d59-kube-api-access-66crd\" (UniqueName: \"kubernetes.io/projected/1ff2c276-f027-4581-94b1-31ce87f60d59-kube-api-access-66crd\") pod \"1ff2c276-f027-4581-94b1-31ce87f60d59\" (UID: \"1ff2c276-f027-4581-94b1-31ce87f60d59\") "
Oct 30 22:15:14.836729 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:14.836723    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1ff2c276-f027-4581-94b1-31ce87f60d59-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1ff2c276-f027-4581-94b1-31ce87f60d59-plugins-config-volume\") pod \"1ff2c276-f027-4581-94b1-31ce87f60d59\" (UID: \"1ff2c276-f027-4581-94b1-31ce87f60d59\") "
Oct 30 22:15:14.837907 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:14.837427    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1ff2c276-f027-4581-94b1-31ce87f60d59-plugins-config-volume" pod "1ff2c276-f027-4581-94b1-31ce87f60d59" (UID: "1ff2c276-f027-4581-94b1-31ce87f60d59"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:15:14.837907 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:14.838039    4473 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dhvz8\" (UniqueName: \"kubernetes.io/projected/3e170928-4f15-43bf-9ad9-1a38b9bb0950-kube-api-access-dhvz8\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:15:14.837907 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:14.838061    4473 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1ff2c276-f027-4581-94b1-31ce87f60d59-plugins-config-volume\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:15:14.844139 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:14.844107    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1ff2c276-f027-4581-94b1-31ce87f60d59-kube-api-access-66crd" pod "1ff2c276-f027-4581-94b1-31ce87f60d59" (UID: "1ff2c276-f027-4581-94b1-31ce87f60d59"). InnerVolumeSpecName "kube-api-access-66crd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:15:14.940358 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:14.940314    4473 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-66crd\" (UniqueName: \"kubernetes.io/projected/1ff2c276-f027-4581-94b1-31ce87f60d59-kube-api-access-66crd\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:15:15.142436 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:15.142406    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-7c826c-9b5d559f7-dr9ng"]
Oct 30 22:15:15.147087 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:15.147061    4473 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-7c826c-9b5d559f7-dr9ng"]
Oct 30 22:15:15.734461 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:15.734433    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-7c826c-86f4595c8c-llmxr"
Oct 30 22:15:15.735009 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:15.734468    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-7c826c-86f4595c8c-llmxr" event={"ID":"1ff2c276-f027-4581-94b1-31ce87f60d59","Type":"ContainerDied","Data":"9a7b5421a0bae2b56a8bead1c975c80b1d7aab19337ea85d8e5a85f4f710fe57"}
Oct 30 22:15:15.735009 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:15.734529    4473 scope.go:122] "RemoveContainer" containerID="dc0af6b693a9399be3a1857b89f03a4f1090502ae70fc9a7bf64716c2017fec5"
Oct 30 22:15:15.904119 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:15.904083    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-7c826c-86f4595c8c-llmxr"]
Oct 30 22:15:15.954097 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:15.954046    4473 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-7c826c-86f4595c8c-llmxr"]
Oct 30 22:15:16.390485 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:16.390454    4473 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1ff2c276-f027-4581-94b1-31ce87f60d59" path="/var/lib/kubelet/pods/1ff2c276-f027-4581-94b1-31ce87f60d59/volumes"
Oct 30 22:15:16.391011 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:16.390993    4473 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3e170928-4f15-43bf-9ad9-1a38b9bb0950" path="/var/lib/kubelet/pods/3e170928-4f15-43bf-9ad9-1a38b9bb0950/volumes"
Oct 30 22:15:19.727221 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:19.727188    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-db492e-676764d957-6j782"
Oct 30 22:15:20.326206 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:15:20.326166    4473 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fncvg" containerName="extractor"
Oct 30 22:15:24.268663 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:24.268623    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-242ddaf0-576ccc4d89-m89lr"]
Oct 30 22:15:24.379667 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:24.379630    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-242ddaf0-7b7b974bc-7q9bh"]
Oct 30 22:15:29.269260 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:29.269210    4473 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-242ddaf0-576ccc4d89-m89lr" podUID="13ab2734-6186-480c-ae76-6f00a295e1e1" containerName="payload-processing" containerID="cri-o://dfac5ec330b025a5580505fb7749c765597cd13f18c3ffb06148e38e54655d7a" gracePeriod=25
Oct 30 22:15:29.380058 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:29.379989    4473 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-242ddaf0-7b7b974bc-7q9bh" podUID="75c6c21f-c5c0-4f9e-9f12-621e01735dda" containerName="payload-pre-processing" containerID="cri-o://d7ee2d15da605bc96fb1440f998fef85d432313e82010bee47ac4cdcf870ffbf" gracePeriod=25
Oct 30 22:15:32.032103 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:32.032066    4473 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-5dff71-5b5d58b5bf-bmk4c"]
Oct 30 22:15:32.032849 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:32.032832    4473 cpu_manager.go:414] "containerMap: removing container" podUID="3e170928-4f15-43bf-9ad9-1a38b9bb0950" containerName="payload-processing"
Oct 30 22:15:32.032893 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:32.032850    4473 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3e170928-4f15-43bf-9ad9-1a38b9bb0950" containerName="payload-processing"
Oct 30 22:15:32.032893 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:32.032869    4473 cpu_manager.go:414] "containerMap: removing container" podUID="dc340a9f-0368-4d8f-9047-be78b84f8a4c" containerName="test-curl-7314-6b259a"
Oct 30 22:15:32.032893 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:32.032874    4473 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="dc340a9f-0368-4d8f-9047-be78b84f8a4c" containerName="test-curl-7314-6b259a"
Oct 30 22:15:32.033028 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:32.032900    4473 cpu_manager.go:414] "containerMap: removing container" podUID="1ff2c276-f027-4581-94b1-31ce87f60d59" containerName="payload-pre-processing"
Oct 30 22:15:32.033028 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:32.032905    4473 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1ff2c276-f027-4581-94b1-31ce87f60d59" containerName="payload-pre-processing"
Oct 30 22:15:32.033093 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:32.033038    4473 memory_manager.go:365] "RemoveStaleState removing state" podUID="1ff2c276-f027-4581-94b1-31ce87f60d59" containerName="payload-pre-processing"
Oct 30 22:15:32.033093 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:32.033052    4473 memory_manager.go:365] "RemoveStaleState removing state" podUID="3e170928-4f15-43bf-9ad9-1a38b9bb0950" containerName="payload-processing"
Oct 30 22:15:32.033093 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:32.033061    4473 memory_manager.go:365] "RemoveStaleState removing state" podUID="dc340a9f-0368-4d8f-9047-be78b84f8a4c" containerName="test-curl-7314-6b259a"
Oct 30 22:15:32.037041 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:32.037024    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-5dff71-5b5d58b5bf-bmk4c"
Oct 30 22:15:32.090325 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:32.090296    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w5-5dff71-serving-cert\""
Oct 30 22:15:32.140330 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:32.140284    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wmpjf\" (UniqueName: \"kubernetes.io/projected/351fd23f-7686-4ce9-903a-95d40b6a521c-kube-api-access-wmpjf\") pod \"maas-api-e2e-shared-b-w5-5dff71-5b5d58b5bf-bmk4c\" (UID: \"351fd23f-7686-4ce9-903a-95d40b6a521c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-5dff71-5b5d58b5bf-bmk4c"
Oct 30 22:15:32.140516 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:32.140363    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/351fd23f-7686-4ce9-903a-95d40b6a521c-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w5-5dff71-5b5d58b5bf-bmk4c\" (UID: \"351fd23f-7686-4ce9-903a-95d40b6a521c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-5dff71-5b5d58b5bf-bmk4c"
Oct 30 22:15:32.140516 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:32.140497    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/351fd23f-7686-4ce9-903a-95d40b6a521c-maas-api-tls\") pod \"maas-api-e2e-shared-b-w5-5dff71-5b5d58b5bf-bmk4c\" (UID: \"351fd23f-7686-4ce9-903a-95d40b6a521c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-5dff71-5b5d58b5bf-bmk4c"
Oct 30 22:15:32.140792 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:32.140701    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/351fd23f-7686-4ce9-903a-95d40b6a521c-metrics-certs\") pod \"maas-api-e2e-shared-b-w5-5dff71-5b5d58b5bf-bmk4c\" (UID: \"351fd23f-7686-4ce9-903a-95d40b6a521c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-5dff71-5b5d58b5bf-bmk4c"
Oct 30 22:15:32.147754 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:32.147713    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-5dff71-5b5d58b5bf-bmk4c"]
Oct 30 22:15:32.242403 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:32.242365    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/351fd23f-7686-4ce9-903a-95d40b6a521c-metrics-certs\") pod \"maas-api-e2e-shared-b-w5-5dff71-5b5d58b5bf-bmk4c\" (UID: \"351fd23f-7686-4ce9-903a-95d40b6a521c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-5dff71-5b5d58b5bf-bmk4c"
Oct 30 22:15:32.242627 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:32.242489    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wmpjf\" (UniqueName: \"kubernetes.io/projected/351fd23f-7686-4ce9-903a-95d40b6a521c-kube-api-access-wmpjf\") pod \"maas-api-e2e-shared-b-w5-5dff71-5b5d58b5bf-bmk4c\" (UID: \"351fd23f-7686-4ce9-903a-95d40b6a521c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-5dff71-5b5d58b5bf-bmk4c"
Oct 30 22:15:32.242627 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:32.242516    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/351fd23f-7686-4ce9-903a-95d40b6a521c-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w5-5dff71-5b5d58b5bf-bmk4c\" (UID: \"351fd23f-7686-4ce9-903a-95d40b6a521c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-5dff71-5b5d58b5bf-bmk4c"
Oct 30 22:15:32.242627 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:32.242567    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/351fd23f-7686-4ce9-903a-95d40b6a521c-maas-api-tls\") pod \"maas-api-e2e-shared-b-w5-5dff71-5b5d58b5bf-bmk4c\" (UID: \"351fd23f-7686-4ce9-903a-95d40b6a521c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-5dff71-5b5d58b5bf-bmk4c"
Oct 30 22:15:32.242822 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:15:32.242680    4473 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-5dff71-serving-cert: secret "maas-api-e2e-shared-b-w5-5dff71-serving-cert" not found
Oct 30 22:15:32.242822 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:15:32.242747    4473 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/351fd23f-7686-4ce9-903a-95d40b6a521c-maas-api-tls podName:351fd23f-7686-4ce9-903a-95d40b6a521c nodeName:}" failed. No retries permitted until 2026-10-30 22:15:32.742728169 +0000 UTC m=+1978.516096651 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/351fd23f-7686-4ce9-903a-95d40b6a521c-maas-api-tls") pod "maas-api-e2e-shared-b-w5-5dff71-5b5d58b5bf-bmk4c" (UID: "351fd23f-7686-4ce9-903a-95d40b6a521c") : secret "maas-api-e2e-shared-b-w5-5dff71-serving-cert" not found
Oct 30 22:15:32.243318 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:32.243298    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/351fd23f-7686-4ce9-903a-95d40b6a521c-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w5-5dff71-5b5d58b5bf-bmk4c\" (UID: \"351fd23f-7686-4ce9-903a-95d40b6a521c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-5dff71-5b5d58b5bf-bmk4c"
Oct 30 22:15:32.245253 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:32.245222    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/351fd23f-7686-4ce9-903a-95d40b6a521c-metrics-certs\") pod \"maas-api-e2e-shared-b-w5-5dff71-5b5d58b5bf-bmk4c\" (UID: \"351fd23f-7686-4ce9-903a-95d40b6a521c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-5dff71-5b5d58b5bf-bmk4c"
Oct 30 22:15:32.301716 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:32.301644    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wmpjf\" (UniqueName: \"kubernetes.io/projected/351fd23f-7686-4ce9-903a-95d40b6a521c-kube-api-access-wmpjf\") pod \"maas-api-e2e-shared-b-w5-5dff71-5b5d58b5bf-bmk4c\" (UID: \"351fd23f-7686-4ce9-903a-95d40b6a521c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-5dff71-5b5d58b5bf-bmk4c"
Oct 30 22:15:32.752166 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:32.752125    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/351fd23f-7686-4ce9-903a-95d40b6a521c-maas-api-tls\") pod \"maas-api-e2e-shared-b-w5-5dff71-5b5d58b5bf-bmk4c\" (UID: \"351fd23f-7686-4ce9-903a-95d40b6a521c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-5dff71-5b5d58b5bf-bmk4c"
Oct 30 22:15:32.755076 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:32.755049    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/351fd23f-7686-4ce9-903a-95d40b6a521c-maas-api-tls\") pod \"maas-api-e2e-shared-b-w5-5dff71-5b5d58b5bf-bmk4c\" (UID: \"351fd23f-7686-4ce9-903a-95d40b6a521c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-5dff71-5b5d58b5bf-bmk4c"
Oct 30 22:15:32.952267 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:32.952165    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-5dff71-5b5d58b5bf-bmk4c"
Oct 30 22:15:33.344399 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:33.344370    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-5dff71-5b5d58b5bf-bmk4c"]
Oct 30 22:15:33.344780 ip-10-0-140-18 kubenswrapper[4473]: W1030 22:15:33.344414    4473 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod351fd23f_7686_4ce9_903a_95d40b6a521c.slice/crio-c65a1206a24c0f0df1265f048b4ae55c2832dae46784859c9c6c2a16f662175e WatchSource:0}: Error finding container c65a1206a24c0f0df1265f048b4ae55c2832dae46784859c9c6c2a16f662175e: Status 404 returned error can't find the container with id c65a1206a24c0f0df1265f048b4ae55c2832dae46784859c9c6c2a16f662175e
Oct 30 22:15:33.951314 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:33.951272    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-5dff71-5b5d58b5bf-bmk4c" event={"ID":"351fd23f-7686-4ce9-903a-95d40b6a521c","Type":"ContainerStarted","Data":"c65a1206a24c0f0df1265f048b4ae55c2832dae46784859c9c6c2a16f662175e"}
Oct 30 22:15:34.962387 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:34.962349    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-5dff71-5b5d58b5bf-bmk4c" event={"ID":"351fd23f-7686-4ce9-903a-95d40b6a521c","Type":"ContainerStarted","Data":"5f526a00948d1c9c9af86a3ade99b822b9fb0790a60358610d25d5796a9fd03f"}
Oct 30 22:15:34.962877 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:34.962409    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-5dff71-5b5d58b5bf-bmk4c"
Oct 30 22:15:35.081409 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:35.081355    4473 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-5dff71-5b5d58b5bf-bmk4c" podStartSLOduration=3.147413227 podStartE2EDuration="4.081340324s" totalImagesPullingTime="933.927097ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:15:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:15:35.080415338 +0000 UTC m=+1980.853783826" watchObservedRunningTime="2026-10-30 22:15:35.081340324 +0000 UTC m=+1980.854708812"
Oct 30 22:15:37.101550 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:37.101520    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-mt-242ddaf0-7b7b974bc-7q9bh" podUID="75c6c21f-c5c0-4f9e-9f12-621e01735dda" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:15:38.100876 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:38.100834    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-mt-242ddaf0-576ccc4d89-m89lr" podUID="13ab2734-6186-480c-ae76-6f00a295e1e1" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:15:41.146847 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:41.146805    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-5dff71-5b5d58b5bf-bmk4c"
Oct 30 22:15:47.100675 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:47.100630    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-mt-242ddaf0-7b7b974bc-7q9bh" podUID="75c6c21f-c5c0-4f9e-9f12-621e01735dda" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:15:48.123805 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:48.123763    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-mt-242ddaf0-576ccc4d89-m89lr" podUID="13ab2734-6186-480c-ae76-6f00a295e1e1" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:15:49.605366 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:49.605336    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-242ddaf0-576ccc4d89-m89lr"
Oct 30 22:15:49.684118 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:49.684089    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-242ddaf0-7b7b974bc-7q9bh"
Oct 30 22:15:49.708843 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:49.708805    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/13ab2734-6186-480c-ae76-6f00a295e1e1-kube-api-access-dqqsd\" (UniqueName: \"kubernetes.io/projected/13ab2734-6186-480c-ae76-6f00a295e1e1-kube-api-access-dqqsd\") pod \"13ab2734-6186-480c-ae76-6f00a295e1e1\" (UID: \"13ab2734-6186-480c-ae76-6f00a295e1e1\") "
Oct 30 22:15:49.709093 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:49.709078    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/13ab2734-6186-480c-ae76-6f00a295e1e1-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/13ab2734-6186-480c-ae76-6f00a295e1e1-plugins-config-volume\") pod \"13ab2734-6186-480c-ae76-6f00a295e1e1\" (UID: \"13ab2734-6186-480c-ae76-6f00a295e1e1\") "
Oct 30 22:15:49.709478 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:49.709453    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/13ab2734-6186-480c-ae76-6f00a295e1e1-plugins-config-volume" pod "13ab2734-6186-480c-ae76-6f00a295e1e1" (UID: "13ab2734-6186-480c-ae76-6f00a295e1e1"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:15:49.709582 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:49.709561    4473 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/13ab2734-6186-480c-ae76-6f00a295e1e1-plugins-config-volume\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:15:49.711389 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:49.711365    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/13ab2734-6186-480c-ae76-6f00a295e1e1-kube-api-access-dqqsd" pod "13ab2734-6186-480c-ae76-6f00a295e1e1" (UID: "13ab2734-6186-480c-ae76-6f00a295e1e1"). InnerVolumeSpecName "kube-api-access-dqqsd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:15:49.810669 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:49.810578    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/75c6c21f-c5c0-4f9e-9f12-621e01735dda-kube-api-access-tdvb5\" (UniqueName: \"kubernetes.io/projected/75c6c21f-c5c0-4f9e-9f12-621e01735dda-kube-api-access-tdvb5\") pod \"75c6c21f-c5c0-4f9e-9f12-621e01735dda\" (UID: \"75c6c21f-c5c0-4f9e-9f12-621e01735dda\") "
Oct 30 22:15:49.810846 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:49.810709    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/75c6c21f-c5c0-4f9e-9f12-621e01735dda-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/75c6c21f-c5c0-4f9e-9f12-621e01735dda-plugins-config-volume\") pod \"75c6c21f-c5c0-4f9e-9f12-621e01735dda\" (UID: \"75c6c21f-c5c0-4f9e-9f12-621e01735dda\") "
Oct 30 22:15:49.811163 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:49.811138    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/75c6c21f-c5c0-4f9e-9f12-621e01735dda-plugins-config-volume" pod "75c6c21f-c5c0-4f9e-9f12-621e01735dda" (UID: "75c6c21f-c5c0-4f9e-9f12-621e01735dda"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:15:49.811258 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:49.811208    4473 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dqqsd\" (UniqueName: \"kubernetes.io/projected/13ab2734-6186-480c-ae76-6f00a295e1e1-kube-api-access-dqqsd\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:15:49.812992 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:49.812910    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/75c6c21f-c5c0-4f9e-9f12-621e01735dda-kube-api-access-tdvb5" pod "75c6c21f-c5c0-4f9e-9f12-621e01735dda" (UID: "75c6c21f-c5c0-4f9e-9f12-621e01735dda"). InnerVolumeSpecName "kube-api-access-tdvb5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:15:49.916841 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:49.913679    4473 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tdvb5\" (UniqueName: \"kubernetes.io/projected/75c6c21f-c5c0-4f9e-9f12-621e01735dda-kube-api-access-tdvb5\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:15:49.916841 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:49.913719    4473 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/75c6c21f-c5c0-4f9e-9f12-621e01735dda-plugins-config-volume\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:15:50.310317 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:50.305587    4473 generic.go:356] "Generic (PLEG): container finished" podID="13ab2734-6186-480c-ae76-6f00a295e1e1" containerID="dfac5ec330b025a5580505fb7749c765597cd13f18c3ffb06148e38e54655d7a" exitCode=0
Oct 30 22:15:50.310317 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:50.305863    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-242ddaf0-576ccc4d89-m89lr" event={"ID":"13ab2734-6186-480c-ae76-6f00a295e1e1","Type":"ContainerDied","Data":"dfac5ec330b025a5580505fb7749c765597cd13f18c3ffb06148e38e54655d7a"}
Oct 30 22:15:50.310317 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:50.306043    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-242ddaf0-576ccc4d89-m89lr" event={"ID":"13ab2734-6186-480c-ae76-6f00a295e1e1","Type":"ContainerDied","Data":"8150cba6f4275de9af661856b9917909059d38735630f363d2e7fa83a42abf8a"}
Oct 30 22:15:50.310317 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:50.306088    4473 scope.go:122] "RemoveContainer" containerID="dfac5ec330b025a5580505fb7749c765597cd13f18c3ffb06148e38e54655d7a"
Oct 30 22:15:50.310317 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:50.306629    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-242ddaf0-576ccc4d89-m89lr"
Oct 30 22:15:50.311032 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:50.310703    4473 generic.go:356] "Generic (PLEG): container finished" podID="75c6c21f-c5c0-4f9e-9f12-621e01735dda" containerID="d7ee2d15da605bc96fb1440f998fef85d432313e82010bee47ac4cdcf870ffbf" exitCode=0
Oct 30 22:15:50.311032 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:50.310863    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-242ddaf0-7b7b974bc-7q9bh" event={"ID":"75c6c21f-c5c0-4f9e-9f12-621e01735dda","Type":"ContainerDied","Data":"d7ee2d15da605bc96fb1440f998fef85d432313e82010bee47ac4cdcf870ffbf"}
Oct 30 22:15:50.311202 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:50.310912    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-242ddaf0-7b7b974bc-7q9bh" event={"ID":"75c6c21f-c5c0-4f9e-9f12-621e01735dda","Type":"ContainerDied","Data":"b7f391698912f145de2270d9c59a07635acb309bdf05845667ae8c5b1a8855cc"}
Oct 30 22:15:50.311325 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:50.311306    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-242ddaf0-7b7b974bc-7q9bh"
Oct 30 22:15:50.327782 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:50.327758    4473 scope.go:122] "RemoveContainer" containerID="dfac5ec330b025a5580505fb7749c765597cd13f18c3ffb06148e38e54655d7a"
Oct 30 22:15:50.328202 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:15:50.328172    4473 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dfac5ec330b025a5580505fb7749c765597cd13f18c3ffb06148e38e54655d7a\": container with ID starting with dfac5ec330b025a5580505fb7749c765597cd13f18c3ffb06148e38e54655d7a not found: ID does not exist" containerID="dfac5ec330b025a5580505fb7749c765597cd13f18c3ffb06148e38e54655d7a"
Oct 30 22:15:50.328319 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:50.328213    4473 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dfac5ec330b025a5580505fb7749c765597cd13f18c3ffb06148e38e54655d7a"} err="failed to get container status \"dfac5ec330b025a5580505fb7749c765597cd13f18c3ffb06148e38e54655d7a\": rpc error: code = NotFound desc = could not find container \"dfac5ec330b025a5580505fb7749c765597cd13f18c3ffb06148e38e54655d7a\": container with ID starting with dfac5ec330b025a5580505fb7749c765597cd13f18c3ffb06148e38e54655d7a not found: ID does not exist"
Oct 30 22:15:50.328319 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:50.328240    4473 scope.go:122] "RemoveContainer" containerID="d7ee2d15da605bc96fb1440f998fef85d432313e82010bee47ac4cdcf870ffbf"
Oct 30 22:15:50.343049 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:50.343028    4473 scope.go:122] "RemoveContainer" containerID="d7ee2d15da605bc96fb1440f998fef85d432313e82010bee47ac4cdcf870ffbf"
Oct 30 22:15:50.343713 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:15:50.343686    4473 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d7ee2d15da605bc96fb1440f998fef85d432313e82010bee47ac4cdcf870ffbf\": container with ID starting with d7ee2d15da605bc96fb1440f998fef85d432313e82010bee47ac4cdcf870ffbf not found: ID does not exist" containerID="d7ee2d15da605bc96fb1440f998fef85d432313e82010bee47ac4cdcf870ffbf"
Oct 30 22:15:50.343801 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:50.343724    4473 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d7ee2d15da605bc96fb1440f998fef85d432313e82010bee47ac4cdcf870ffbf"} err="failed to get container status \"d7ee2d15da605bc96fb1440f998fef85d432313e82010bee47ac4cdcf870ffbf\": rpc error: code = NotFound desc = could not find container \"d7ee2d15da605bc96fb1440f998fef85d432313e82010bee47ac4cdcf870ffbf\": container with ID starting with d7ee2d15da605bc96fb1440f998fef85d432313e82010bee47ac4cdcf870ffbf not found: ID does not exist"
Oct 30 22:15:50.353903 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:15:50.353539    4473 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fncvg" containerName="extractor"
Oct 30 22:15:50.479436 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:50.479390    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-242ddaf0-576ccc4d89-m89lr"]
Oct 30 22:15:50.482977 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:50.482951    4473 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-242ddaf0-576ccc4d89-m89lr"]
Oct 30 22:15:50.597430 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:50.597344    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-242ddaf0-7b7b974bc-7q9bh"]
Oct 30 22:15:50.603381 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:50.603352    4473 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-242ddaf0-7b7b974bc-7q9bh"]
Oct 30 22:15:52.359014 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:52.358984    4473 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="13ab2734-6186-480c-ae76-6f00a295e1e1" path="/var/lib/kubelet/pods/13ab2734-6186-480c-ae76-6f00a295e1e1/volumes"
Oct 30 22:15:52.359467 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:52.359452    4473 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="75c6c21f-c5c0-4f9e-9f12-621e01735dda" path="/var/lib/kubelet/pods/75c6c21f-c5c0-4f9e-9f12-621e01735dda/volumes"
Oct 30 22:15:56.997976 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:56.997908    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w6-5d36f6-84c6b5685c-zjsct"]
Oct 30 22:15:57.108267 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:57.108236    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w6-5d36f6-757f4b4d99-mrs2n"]
Oct 30 22:15:57.750625 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:57.750586    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-8f1d78db-openshift-default-6bcc897b9-r5blj"]
Oct 30 22:15:57.750882 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:15:57.750853    4473 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-8f1d78db-openshift-default-6bcc897b9-r5blj" podUID="3e4b6a7f-8bfd-49d8-be83-674d091c99d4" containerName="istio-proxy" containerID="cri-o://3efab88c68c5ee507793ab011d2598e5419f1bed066fba036f9cffd091450550" gracePeriod=30
Oct 30 22:16:00.614468 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:00.614426    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-2lzcz"]
Oct 30 22:16:00.614991 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:00.614683    4473 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-2lzcz" podUID="83504be1-81ee-4a97-93e3-a61ecb2f3b2d" containerName="istio-proxy" containerID="cri-o://16582c30f5a2afae5aac523619aa462ac28bc01d142bdc17c2215fde23c9d47e" gracePeriod=30
Oct 30 22:16:01.998455 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:01.998405    4473 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w6-5d36f6-84c6b5685c-zjsct" podUID="b72da9e3-1c30-4fb9-a879-1279c67a7453" containerName="payload-processing" containerID="cri-o://2d0ef0b34a04084a97c378fc1accfc726888fa1bfe96cffeb1b0678f3425344a" gracePeriod=25
Oct 30 22:16:02.108785 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:02.108732    4473 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w6-5d36f6-757f4b4d99-mrs2n" podUID="2810c233-3078-4bc0-bb3e-bc863e7786d2" containerName="payload-pre-processing" containerID="cri-o://6454d5d1253d3fd12a27db76a7970982c6e6f74aae1343667872ba420b03d185" gracePeriod=25
Oct 30 22:16:02.160800 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:02.160713    4473 patch_prober.go:29] interesting pod/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-2lzcz container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 22:16:02.160998 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:02.160818    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-2lzcz" podUID="83504be1-81ee-4a97-93e3-a61ecb2f3b2d" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 22:16:02.307880 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:02.307488    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w6-5d36f6-84c6b5685c-zjsct"
Oct 30 22:16:02.356850 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:02.355513    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b72da9e3-1c30-4fb9-a879-1279c67a7453-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b72da9e3-1c30-4fb9-a879-1279c67a7453-plugins-config-volume\") pod \"b72da9e3-1c30-4fb9-a879-1279c67a7453\" (UID: \"b72da9e3-1c30-4fb9-a879-1279c67a7453\") "
Oct 30 22:16:02.356850 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:02.355598    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b72da9e3-1c30-4fb9-a879-1279c67a7453-kube-api-access-b69j9\" (UniqueName: \"kubernetes.io/projected/b72da9e3-1c30-4fb9-a879-1279c67a7453-kube-api-access-b69j9\") pod \"b72da9e3-1c30-4fb9-a879-1279c67a7453\" (UID: \"b72da9e3-1c30-4fb9-a879-1279c67a7453\") "
Oct 30 22:16:02.356850 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:02.356372    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b72da9e3-1c30-4fb9-a879-1279c67a7453-plugins-config-volume" pod "b72da9e3-1c30-4fb9-a879-1279c67a7453" (UID: "b72da9e3-1c30-4fb9-a879-1279c67a7453"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:16:02.363036 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:02.362995    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b72da9e3-1c30-4fb9-a879-1279c67a7453-kube-api-access-b69j9" pod "b72da9e3-1c30-4fb9-a879-1279c67a7453" (UID: "b72da9e3-1c30-4fb9-a879-1279c67a7453"). InnerVolumeSpecName "kube-api-access-b69j9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:16:02.399716 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:02.399689    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w6-5d36f6-757f4b4d99-mrs2n"
Oct 30 22:16:02.407766 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:02.407735    4473 generic.go:356] "Generic (PLEG): container finished" podID="b72da9e3-1c30-4fb9-a879-1279c67a7453" containerID="2d0ef0b34a04084a97c378fc1accfc726888fa1bfe96cffeb1b0678f3425344a" exitCode=0
Oct 30 22:16:02.407977 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:02.407781    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w6-5d36f6-84c6b5685c-zjsct"
Oct 30 22:16:02.407977 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:02.407822    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w6-5d36f6-84c6b5685c-zjsct" event={"ID":"b72da9e3-1c30-4fb9-a879-1279c67a7453","Type":"ContainerDied","Data":"2d0ef0b34a04084a97c378fc1accfc726888fa1bfe96cffeb1b0678f3425344a"}
Oct 30 22:16:02.407977 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:02.407876    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w6-5d36f6-84c6b5685c-zjsct" event={"ID":"b72da9e3-1c30-4fb9-a879-1279c67a7453","Type":"ContainerDied","Data":"86b0aa016b4a7ead5dbf393b84b69e13456f4409eadecc3e4658efa59979d88e"}
Oct 30 22:16:02.407977 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:02.407899    4473 scope.go:122] "RemoveContainer" containerID="2d0ef0b34a04084a97c378fc1accfc726888fa1bfe96cffeb1b0678f3425344a"
Oct 30 22:16:02.409574 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:02.409552    4473 generic.go:356] "Generic (PLEG): container finished" podID="2810c233-3078-4bc0-bb3e-bc863e7786d2" containerID="6454d5d1253d3fd12a27db76a7970982c6e6f74aae1343667872ba420b03d185" exitCode=0
Oct 30 22:16:02.409709 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:02.409602    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w6-5d36f6-757f4b4d99-mrs2n"
Oct 30 22:16:02.409709 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:02.409639    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w6-5d36f6-757f4b4d99-mrs2n" event={"ID":"2810c233-3078-4bc0-bb3e-bc863e7786d2","Type":"ContainerDied","Data":"6454d5d1253d3fd12a27db76a7970982c6e6f74aae1343667872ba420b03d185"}
Oct 30 22:16:02.409709 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:02.409693    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w6-5d36f6-757f4b4d99-mrs2n" event={"ID":"2810c233-3078-4bc0-bb3e-bc863e7786d2","Type":"ContainerDied","Data":"f102c7eac367dc1592c6ac95c61716ea1de275dc29a7975057474ea86a92e553"}
Oct 30 22:16:02.419383 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:02.419367    4473 scope.go:122] "RemoveContainer" containerID="2d0ef0b34a04084a97c378fc1accfc726888fa1bfe96cffeb1b0678f3425344a"
Oct 30 22:16:02.419704 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:16:02.419684    4473 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2d0ef0b34a04084a97c378fc1accfc726888fa1bfe96cffeb1b0678f3425344a\": container with ID starting with 2d0ef0b34a04084a97c378fc1accfc726888fa1bfe96cffeb1b0678f3425344a not found: ID does not exist" containerID="2d0ef0b34a04084a97c378fc1accfc726888fa1bfe96cffeb1b0678f3425344a"
Oct 30 22:16:02.419758 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:02.419713    4473 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2d0ef0b34a04084a97c378fc1accfc726888fa1bfe96cffeb1b0678f3425344a"} err="failed to get container status \"2d0ef0b34a04084a97c378fc1accfc726888fa1bfe96cffeb1b0678f3425344a\": rpc error: code = NotFound desc = could not find container \"2d0ef0b34a04084a97c378fc1accfc726888fa1bfe96cffeb1b0678f3425344a\": container with ID starting with 2d0ef0b34a04084a97c378fc1accfc726888fa1bfe96cffeb1b0678f3425344a not found: ID does not exist"
Oct 30 22:16:02.419758 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:02.419731    4473 scope.go:122] "RemoveContainer" containerID="6454d5d1253d3fd12a27db76a7970982c6e6f74aae1343667872ba420b03d185"
Oct 30 22:16:02.436837 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:02.436800    4473 scope.go:122] "RemoveContainer" containerID="6454d5d1253d3fd12a27db76a7970982c6e6f74aae1343667872ba420b03d185"
Oct 30 22:16:02.437518 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:16:02.437487    4473 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6454d5d1253d3fd12a27db76a7970982c6e6f74aae1343667872ba420b03d185\": container with ID starting with 6454d5d1253d3fd12a27db76a7970982c6e6f74aae1343667872ba420b03d185 not found: ID does not exist" containerID="6454d5d1253d3fd12a27db76a7970982c6e6f74aae1343667872ba420b03d185"
Oct 30 22:16:02.437645 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:02.437526    4473 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6454d5d1253d3fd12a27db76a7970982c6e6f74aae1343667872ba420b03d185"} err="failed to get container status \"6454d5d1253d3fd12a27db76a7970982c6e6f74aae1343667872ba420b03d185\": rpc error: code = NotFound desc = could not find container \"6454d5d1253d3fd12a27db76a7970982c6e6f74aae1343667872ba420b03d185\": container with ID starting with 6454d5d1253d3fd12a27db76a7970982c6e6f74aae1343667872ba420b03d185 not found: ID does not exist"
Oct 30 22:16:02.457190 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:02.457156    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2810c233-3078-4bc0-bb3e-bc863e7786d2-kube-api-access-f852v\" (UniqueName: \"kubernetes.io/projected/2810c233-3078-4bc0-bb3e-bc863e7786d2-kube-api-access-f852v\") pod \"2810c233-3078-4bc0-bb3e-bc863e7786d2\" (UID: \"2810c233-3078-4bc0-bb3e-bc863e7786d2\") "
Oct 30 22:16:02.457390 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:02.457214    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2810c233-3078-4bc0-bb3e-bc863e7786d2-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2810c233-3078-4bc0-bb3e-bc863e7786d2-plugins-config-volume\") pod \"2810c233-3078-4bc0-bb3e-bc863e7786d2\" (UID: \"2810c233-3078-4bc0-bb3e-bc863e7786d2\") "
Oct 30 22:16:02.457781 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:02.457739    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2810c233-3078-4bc0-bb3e-bc863e7786d2-plugins-config-volume" pod "2810c233-3078-4bc0-bb3e-bc863e7786d2" (UID: "2810c233-3078-4bc0-bb3e-bc863e7786d2"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:16:02.458381 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:02.458345    4473 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b72da9e3-1c30-4fb9-a879-1279c67a7453-plugins-config-volume\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:16:02.458381 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:02.458376    4473 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b69j9\" (UniqueName: \"kubernetes.io/projected/b72da9e3-1c30-4fb9-a879-1279c67a7453-kube-api-access-b69j9\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:16:02.458575 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:02.458392    4473 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2810c233-3078-4bc0-bb3e-bc863e7786d2-plugins-config-volume\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:16:02.459946 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:02.459903    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2810c233-3078-4bc0-bb3e-bc863e7786d2-kube-api-access-f852v" pod "2810c233-3078-4bc0-bb3e-bc863e7786d2" (UID: "2810c233-3078-4bc0-bb3e-bc863e7786d2"). InnerVolumeSpecName "kube-api-access-f852v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:16:02.541330 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:02.541245    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w6-5d36f6-84c6b5685c-zjsct"]
Oct 30 22:16:02.545220 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:02.545192    4473 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w6-5d36f6-84c6b5685c-zjsct"]
Oct 30 22:16:02.559961 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:02.559909    4473 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f852v\" (UniqueName: \"kubernetes.io/projected/2810c233-3078-4bc0-bb3e-bc863e7786d2-kube-api-access-f852v\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:16:02.836282 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:02.836192    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w6-5d36f6-757f4b4d99-mrs2n"]
Oct 30 22:16:02.841646 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:02.841618    4473 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w6-5d36f6-757f4b4d99-mrs2n"]
Oct 30 22:16:03.032727 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:03.032702    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-8f1d78db-openshift-default-6bcc897b9-r5blj"
Oct 30 22:16:03.169400 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:03.169362    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3e4b6a7f-8bfd-49d8-be83-674d091c99d4-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3e4b6a7f-8bfd-49d8-be83-674d091c99d4-istio-envoy\") pod \"3e4b6a7f-8bfd-49d8-be83-674d091c99d4\" (UID: \"3e4b6a7f-8bfd-49d8-be83-674d091c99d4\") "
Oct 30 22:16:03.169578 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:03.169486    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/3e4b6a7f-8bfd-49d8-be83-674d091c99d4-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3e4b6a7f-8bfd-49d8-be83-674d091c99d4-istio-podinfo\") pod \"3e4b6a7f-8bfd-49d8-be83-674d091c99d4\" (UID: \"3e4b6a7f-8bfd-49d8-be83-674d091c99d4\") "
Oct 30 22:16:03.169578 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:03.169519    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3e4b6a7f-8bfd-49d8-be83-674d091c99d4-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3e4b6a7f-8bfd-49d8-be83-674d091c99d4-workload-certs\") pod \"3e4b6a7f-8bfd-49d8-be83-674d091c99d4\" (UID: \"3e4b6a7f-8bfd-49d8-be83-674d091c99d4\") "
Oct 30 22:16:03.169721 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:03.169583    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3e4b6a7f-8bfd-49d8-be83-674d091c99d4-kube-api-access-j4247\" (UniqueName: \"kubernetes.io/projected/3e4b6a7f-8bfd-49d8-be83-674d091c99d4-kube-api-access-j4247\") pod \"3e4b6a7f-8bfd-49d8-be83-674d091c99d4\" (UID: \"3e4b6a7f-8bfd-49d8-be83-674d091c99d4\") "
Oct 30 22:16:03.169721 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:03.169649    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3e4b6a7f-8bfd-49d8-be83-674d091c99d4-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3e4b6a7f-8bfd-49d8-be83-674d091c99d4-istio-data\") pod \"3e4b6a7f-8bfd-49d8-be83-674d091c99d4\" (UID: \"3e4b6a7f-8bfd-49d8-be83-674d091c99d4\") "
Oct 30 22:16:03.169831 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:03.169814    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3e4b6a7f-8bfd-49d8-be83-674d091c99d4-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e4b6a7f-8bfd-49d8-be83-674d091c99d4-credential-socket\") pod \"3e4b6a7f-8bfd-49d8-be83-674d091c99d4\" (UID: \"3e4b6a7f-8bfd-49d8-be83-674d091c99d4\") "
Oct 30 22:16:03.169889 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:03.169851    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3e4b6a7f-8bfd-49d8-be83-674d091c99d4-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e4b6a7f-8bfd-49d8-be83-674d091c99d4-workload-socket\") pod \"3e4b6a7f-8bfd-49d8-be83-674d091c99d4\" (UID: \"3e4b6a7f-8bfd-49d8-be83-674d091c99d4\") "
Oct 30 22:16:03.169973 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:03.169897    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3e4b6a7f-8bfd-49d8-be83-674d091c99d4-workload-certs" pod "3e4b6a7f-8bfd-49d8-be83-674d091c99d4" (UID: "3e4b6a7f-8bfd-49d8-be83-674d091c99d4"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:16:03.169973 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:03.169910    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3e4b6a7f-8bfd-49d8-be83-674d091c99d4-istio-token\" (UniqueName: \"kubernetes.io/projected/3e4b6a7f-8bfd-49d8-be83-674d091c99d4-istio-token\") pod \"3e4b6a7f-8bfd-49d8-be83-674d091c99d4\" (UID: \"3e4b6a7f-8bfd-49d8-be83-674d091c99d4\") "
Oct 30 22:16:03.170154 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:03.170131    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3e4b6a7f-8bfd-49d8-be83-674d091c99d4-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3e4b6a7f-8bfd-49d8-be83-674d091c99d4-istiod-ca-cert\") pod \"3e4b6a7f-8bfd-49d8-be83-674d091c99d4\" (UID: \"3e4b6a7f-8bfd-49d8-be83-674d091c99d4\") "
Oct 30 22:16:03.170823 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:03.170127    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3e4b6a7f-8bfd-49d8-be83-674d091c99d4-istio-data" pod "3e4b6a7f-8bfd-49d8-be83-674d091c99d4" (UID: "3e4b6a7f-8bfd-49d8-be83-674d091c99d4"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:16:03.170823 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:03.170304    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3e4b6a7f-8bfd-49d8-be83-674d091c99d4-workload-socket" pod "3e4b6a7f-8bfd-49d8-be83-674d091c99d4" (UID: "3e4b6a7f-8bfd-49d8-be83-674d091c99d4"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:16:03.170823 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:03.170766    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3e4b6a7f-8bfd-49d8-be83-674d091c99d4-istiod-ca-cert" pod "3e4b6a7f-8bfd-49d8-be83-674d091c99d4" (UID: "3e4b6a7f-8bfd-49d8-be83-674d091c99d4"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:16:03.172027 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:03.171348    4473 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e4b6a7f-8bfd-49d8-be83-674d091c99d4-workload-socket\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:16:03.172027 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:03.171377    4473 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3e4b6a7f-8bfd-49d8-be83-674d091c99d4-istiod-ca-cert\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:16:03.172027 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:03.171392    4473 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3e4b6a7f-8bfd-49d8-be83-674d091c99d4-workload-certs\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:16:03.172027 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:03.171406    4473 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3e4b6a7f-8bfd-49d8-be83-674d091c99d4-istio-data\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:16:03.173348 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:03.173307    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3e4b6a7f-8bfd-49d8-be83-674d091c99d4-istio-token" pod "3e4b6a7f-8bfd-49d8-be83-674d091c99d4" (UID: "3e4b6a7f-8bfd-49d8-be83-674d091c99d4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:16:03.173462 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:03.173438    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3e4b6a7f-8bfd-49d8-be83-674d091c99d4-istio-envoy" pod "3e4b6a7f-8bfd-49d8-be83-674d091c99d4" (UID: "3e4b6a7f-8bfd-49d8-be83-674d091c99d4"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:16:03.176277 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:03.176247    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3e4b6a7f-8bfd-49d8-be83-674d091c99d4-credential-socket" pod "3e4b6a7f-8bfd-49d8-be83-674d091c99d4" (UID: "3e4b6a7f-8bfd-49d8-be83-674d091c99d4"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:16:03.176852 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:03.176831    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3e4b6a7f-8bfd-49d8-be83-674d091c99d4-istio-podinfo" pod "3e4b6a7f-8bfd-49d8-be83-674d091c99d4" (UID: "3e4b6a7f-8bfd-49d8-be83-674d091c99d4"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:16:03.177066 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:03.177042    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3e4b6a7f-8bfd-49d8-be83-674d091c99d4-kube-api-access-j4247" pod "3e4b6a7f-8bfd-49d8-be83-674d091c99d4" (UID: "3e4b6a7f-8bfd-49d8-be83-674d091c99d4"). InnerVolumeSpecName "kube-api-access-j4247". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:16:03.273274 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:03.273235    4473 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e4b6a7f-8bfd-49d8-be83-674d091c99d4-credential-socket\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:16:03.273274 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:03.273269    4473 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3e4b6a7f-8bfd-49d8-be83-674d091c99d4-istio-token\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:16:03.273498 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:03.273295    4473 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3e4b6a7f-8bfd-49d8-be83-674d091c99d4-istio-envoy\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:16:03.273498 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:03.273305    4473 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3e4b6a7f-8bfd-49d8-be83-674d091c99d4-istio-podinfo\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:16:03.273498 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:03.273314    4473 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j4247\" (UniqueName: \"kubernetes.io/projected/3e4b6a7f-8bfd-49d8-be83-674d091c99d4-kube-api-access-j4247\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:16:03.419316 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:03.419280    4473 generic.go:356] "Generic (PLEG): container finished" podID="3e4b6a7f-8bfd-49d8-be83-674d091c99d4" containerID="3efab88c68c5ee507793ab011d2598e5419f1bed066fba036f9cffd091450550" exitCode=0
Oct 30 22:16:03.419531 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:03.419356    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-8f1d78db-openshift-default-6bcc897b9-r5blj"
Oct 30 22:16:03.419531 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:03.419369    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-8f1d78db-openshift-default-6bcc897b9-r5blj" event={"ID":"3e4b6a7f-8bfd-49d8-be83-674d091c99d4","Type":"ContainerDied","Data":"3efab88c68c5ee507793ab011d2598e5419f1bed066fba036f9cffd091450550"}
Oct 30 22:16:03.419531 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:03.419434    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-8f1d78db-openshift-default-6bcc897b9-r5blj" event={"ID":"3e4b6a7f-8bfd-49d8-be83-674d091c99d4","Type":"ContainerDied","Data":"a09ace75c64f3898543a2476d7fb7d0b946519493530ba3ab304a772d22ae09a"}
Oct 30 22:16:03.419531 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:03.419458    4473 scope.go:122] "RemoveContainer" containerID="3efab88c68c5ee507793ab011d2598e5419f1bed066fba036f9cffd091450550"
Oct 30 22:16:03.432427 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:03.432404    4473 scope.go:122] "RemoveContainer" containerID="99754613fc190c15476ba88cb6724abf21129e7aa43928ceaee60e7bc615d280"
Oct 30 22:16:03.443822 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:03.443799    4473 scope.go:122] "RemoveContainer" containerID="3efab88c68c5ee507793ab011d2598e5419f1bed066fba036f9cffd091450550"
Oct 30 22:16:03.444226 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:16:03.444205    4473 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3efab88c68c5ee507793ab011d2598e5419f1bed066fba036f9cffd091450550\": container with ID starting with 3efab88c68c5ee507793ab011d2598e5419f1bed066fba036f9cffd091450550 not found: ID does not exist" containerID="3efab88c68c5ee507793ab011d2598e5419f1bed066fba036f9cffd091450550"
Oct 30 22:16:03.444297 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:03.444237    4473 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3efab88c68c5ee507793ab011d2598e5419f1bed066fba036f9cffd091450550"} err="failed to get container status \"3efab88c68c5ee507793ab011d2598e5419f1bed066fba036f9cffd091450550\": rpc error: code = NotFound desc = could not find container \"3efab88c68c5ee507793ab011d2598e5419f1bed066fba036f9cffd091450550\": container with ID starting with 3efab88c68c5ee507793ab011d2598e5419f1bed066fba036f9cffd091450550 not found: ID does not exist"
Oct 30 22:16:03.444297 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:03.444256    4473 scope.go:122] "RemoveContainer" containerID="99754613fc190c15476ba88cb6724abf21129e7aa43928ceaee60e7bc615d280"
Oct 30 22:16:03.444523 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:16:03.444505    4473 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"99754613fc190c15476ba88cb6724abf21129e7aa43928ceaee60e7bc615d280\": container with ID starting with 99754613fc190c15476ba88cb6724abf21129e7aa43928ceaee60e7bc615d280 not found: ID does not exist" containerID="99754613fc190c15476ba88cb6724abf21129e7aa43928ceaee60e7bc615d280"
Oct 30 22:16:03.444562 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:03.444531    4473 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"99754613fc190c15476ba88cb6724abf21129e7aa43928ceaee60e7bc615d280"} err="failed to get container status \"99754613fc190c15476ba88cb6724abf21129e7aa43928ceaee60e7bc615d280\": rpc error: code = NotFound desc = could not find container \"99754613fc190c15476ba88cb6724abf21129e7aa43928ceaee60e7bc615d280\": container with ID starting with 99754613fc190c15476ba88cb6724abf21129e7aa43928ceaee60e7bc615d280 not found: ID does not exist"
Oct 30 22:16:03.587994 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:03.587954    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-8f1d78db-openshift-default-6bcc897b9-r5blj"]
Oct 30 22:16:03.639152 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:03.639116    4473 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-8f1d78db-openshift-default-6bcc897b9-r5blj"]
Oct 30 22:16:04.358931 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:04.358883    4473 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2810c233-3078-4bc0-bb3e-bc863e7786d2" path="/var/lib/kubelet/pods/2810c233-3078-4bc0-bb3e-bc863e7786d2/volumes"
Oct 30 22:16:04.359447 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:04.359421    4473 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3e4b6a7f-8bfd-49d8-be83-674d091c99d4" path="/var/lib/kubelet/pods/3e4b6a7f-8bfd-49d8-be83-674d091c99d4/volumes"
Oct 30 22:16:04.359863 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:04.359849    4473 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b72da9e3-1c30-4fb9-a879-1279c67a7453" path="/var/lib/kubelet/pods/b72da9e3-1c30-4fb9-a879-1279c67a7453/volumes"
Oct 30 22:16:05.902557 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:05.902493    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-2lzcz"
Oct 30 22:16:06.014636 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:06.014602    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/83504be1-81ee-4a97-93e3-a61ecb2f3b2d-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/83504be1-81ee-4a97-93e3-a61ecb2f3b2d-workload-socket\") pod \"83504be1-81ee-4a97-93e3-a61ecb2f3b2d\" (UID: \"83504be1-81ee-4a97-93e3-a61ecb2f3b2d\") "
Oct 30 22:16:06.014846 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:06.014678    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/83504be1-81ee-4a97-93e3-a61ecb2f3b2d-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/83504be1-81ee-4a97-93e3-a61ecb2f3b2d-istiod-ca-cert\") pod \"83504be1-81ee-4a97-93e3-a61ecb2f3b2d\" (UID: \"83504be1-81ee-4a97-93e3-a61ecb2f3b2d\") "
Oct 30 22:16:06.014846 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:06.014698    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/83504be1-81ee-4a97-93e3-a61ecb2f3b2d-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/83504be1-81ee-4a97-93e3-a61ecb2f3b2d-workload-certs\") pod \"83504be1-81ee-4a97-93e3-a61ecb2f3b2d\" (UID: \"83504be1-81ee-4a97-93e3-a61ecb2f3b2d\") "
Oct 30 22:16:06.014846 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:06.014758    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/83504be1-81ee-4a97-93e3-a61ecb2f3b2d-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/83504be1-81ee-4a97-93e3-a61ecb2f3b2d-istio-data\") pod \"83504be1-81ee-4a97-93e3-a61ecb2f3b2d\" (UID: \"83504be1-81ee-4a97-93e3-a61ecb2f3b2d\") "
Oct 30 22:16:06.014846 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:06.014813    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/83504be1-81ee-4a97-93e3-a61ecb2f3b2d-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/83504be1-81ee-4a97-93e3-a61ecb2f3b2d-istio-podinfo\") pod \"83504be1-81ee-4a97-93e3-a61ecb2f3b2d\" (UID: \"83504be1-81ee-4a97-93e3-a61ecb2f3b2d\") "
Oct 30 22:16:06.014846 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:06.014845    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/83504be1-81ee-4a97-93e3-a61ecb2f3b2d-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/83504be1-81ee-4a97-93e3-a61ecb2f3b2d-istio-envoy\") pod \"83504be1-81ee-4a97-93e3-a61ecb2f3b2d\" (UID: \"83504be1-81ee-4a97-93e3-a61ecb2f3b2d\") "
Oct 30 22:16:06.015408 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:06.014981    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/83504be1-81ee-4a97-93e3-a61ecb2f3b2d-istio-token\" (UniqueName: \"kubernetes.io/projected/83504be1-81ee-4a97-93e3-a61ecb2f3b2d-istio-token\") pod \"83504be1-81ee-4a97-93e3-a61ecb2f3b2d\" (UID: \"83504be1-81ee-4a97-93e3-a61ecb2f3b2d\") "
Oct 30 22:16:06.015408 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:06.015092    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/83504be1-81ee-4a97-93e3-a61ecb2f3b2d-kube-api-access-6869g\" (UniqueName: \"kubernetes.io/projected/83504be1-81ee-4a97-93e3-a61ecb2f3b2d-kube-api-access-6869g\") pod \"83504be1-81ee-4a97-93e3-a61ecb2f3b2d\" (UID: \"83504be1-81ee-4a97-93e3-a61ecb2f3b2d\") "
Oct 30 22:16:06.015408 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:06.015133    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/83504be1-81ee-4a97-93e3-a61ecb2f3b2d-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/83504be1-81ee-4a97-93e3-a61ecb2f3b2d-credential-socket\") pod \"83504be1-81ee-4a97-93e3-a61ecb2f3b2d\" (UID: \"83504be1-81ee-4a97-93e3-a61ecb2f3b2d\") "
Oct 30 22:16:06.015963 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:06.015898    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83504be1-81ee-4a97-93e3-a61ecb2f3b2d-istio-data" pod "83504be1-81ee-4a97-93e3-a61ecb2f3b2d" (UID: "83504be1-81ee-4a97-93e3-a61ecb2f3b2d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:16:06.016160 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:06.016136    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83504be1-81ee-4a97-93e3-a61ecb2f3b2d-credential-socket" pod "83504be1-81ee-4a97-93e3-a61ecb2f3b2d" (UID: "83504be1-81ee-4a97-93e3-a61ecb2f3b2d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:16:06.016295 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:06.016271    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83504be1-81ee-4a97-93e3-a61ecb2f3b2d-workload-socket" pod "83504be1-81ee-4a97-93e3-a61ecb2f3b2d" (UID: "83504be1-81ee-4a97-93e3-a61ecb2f3b2d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:16:06.016360 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:06.016349    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/83504be1-81ee-4a97-93e3-a61ecb2f3b2d-istiod-ca-cert" pod "83504be1-81ee-4a97-93e3-a61ecb2f3b2d" (UID: "83504be1-81ee-4a97-93e3-a61ecb2f3b2d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:16:06.016551 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:06.016528    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83504be1-81ee-4a97-93e3-a61ecb2f3b2d-workload-certs" pod "83504be1-81ee-4a97-93e3-a61ecb2f3b2d" (UID: "83504be1-81ee-4a97-93e3-a61ecb2f3b2d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:16:06.018773 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:06.018749    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/83504be1-81ee-4a97-93e3-a61ecb2f3b2d-kube-api-access-6869g" pod "83504be1-81ee-4a97-93e3-a61ecb2f3b2d" (UID: "83504be1-81ee-4a97-93e3-a61ecb2f3b2d"). InnerVolumeSpecName "kube-api-access-6869g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:16:06.019359 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:06.019336    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83504be1-81ee-4a97-93e3-a61ecb2f3b2d-istio-envoy" pod "83504be1-81ee-4a97-93e3-a61ecb2f3b2d" (UID: "83504be1-81ee-4a97-93e3-a61ecb2f3b2d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:16:06.020344 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:06.020308    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/83504be1-81ee-4a97-93e3-a61ecb2f3b2d-istio-token" pod "83504be1-81ee-4a97-93e3-a61ecb2f3b2d" (UID: "83504be1-81ee-4a97-93e3-a61ecb2f3b2d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:16:06.023315 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:06.023293    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/83504be1-81ee-4a97-93e3-a61ecb2f3b2d-istio-podinfo" pod "83504be1-81ee-4a97-93e3-a61ecb2f3b2d" (UID: "83504be1-81ee-4a97-93e3-a61ecb2f3b2d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:16:06.117567 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:06.117528    4473 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6869g\" (UniqueName: \"kubernetes.io/projected/83504be1-81ee-4a97-93e3-a61ecb2f3b2d-kube-api-access-6869g\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:16:06.117567 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:06.117562    4473 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/83504be1-81ee-4a97-93e3-a61ecb2f3b2d-credential-socket\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:16:06.117567 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:06.117572    4473 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/83504be1-81ee-4a97-93e3-a61ecb2f3b2d-workload-socket\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:16:06.117822 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:06.117581    4473 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/83504be1-81ee-4a97-93e3-a61ecb2f3b2d-istiod-ca-cert\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:16:06.117822 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:06.117591    4473 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/83504be1-81ee-4a97-93e3-a61ecb2f3b2d-workload-certs\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:16:06.117822 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:06.117601    4473 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/83504be1-81ee-4a97-93e3-a61ecb2f3b2d-istio-data\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:16:06.117822 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:06.117608    4473 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/83504be1-81ee-4a97-93e3-a61ecb2f3b2d-istio-podinfo\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:16:06.117822 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:06.117618    4473 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/83504be1-81ee-4a97-93e3-a61ecb2f3b2d-istio-envoy\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:16:06.117822 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:06.117627    4473 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/83504be1-81ee-4a97-93e3-a61ecb2f3b2d-istio-token\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:16:06.486946 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:06.481875    4473 generic.go:356] "Generic (PLEG): container finished" podID="83504be1-81ee-4a97-93e3-a61ecb2f3b2d" containerID="16582c30f5a2afae5aac523619aa462ac28bc01d142bdc17c2215fde23c9d47e" exitCode=0
Oct 30 22:16:06.486946 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:06.481958    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-2lzcz" event={"ID":"83504be1-81ee-4a97-93e3-a61ecb2f3b2d","Type":"ContainerDied","Data":"16582c30f5a2afae5aac523619aa462ac28bc01d142bdc17c2215fde23c9d47e"}
Oct 30 22:16:06.486946 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:06.482028    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-2lzcz" event={"ID":"83504be1-81ee-4a97-93e3-a61ecb2f3b2d","Type":"ContainerDied","Data":"54da5642fe8c061b3b5372de248e911939b297eccb115b71903371ed0069a407"}
Oct 30 22:16:06.486946 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:06.482060    4473 scope.go:122] "RemoveContainer" containerID="16582c30f5a2afae5aac523619aa462ac28bc01d142bdc17c2215fde23c9d47e"
Oct 30 22:16:06.486946 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:06.482188    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-2lzcz"
Oct 30 22:16:06.512022 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:06.511612    4473 scope.go:122] "RemoveContainer" containerID="16582c30f5a2afae5aac523619aa462ac28bc01d142bdc17c2215fde23c9d47e"
Oct 30 22:16:06.512380 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:16:06.512259    4473 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"16582c30f5a2afae5aac523619aa462ac28bc01d142bdc17c2215fde23c9d47e\": container with ID starting with 16582c30f5a2afae5aac523619aa462ac28bc01d142bdc17c2215fde23c9d47e not found: ID does not exist" containerID="16582c30f5a2afae5aac523619aa462ac28bc01d142bdc17c2215fde23c9d47e"
Oct 30 22:16:06.512380 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:06.512299    4473 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"16582c30f5a2afae5aac523619aa462ac28bc01d142bdc17c2215fde23c9d47e"} err="failed to get container status \"16582c30f5a2afae5aac523619aa462ac28bc01d142bdc17c2215fde23c9d47e\": rpc error: code = NotFound desc = could not find container \"16582c30f5a2afae5aac523619aa462ac28bc01d142bdc17c2215fde23c9d47e\": container with ID starting with 16582c30f5a2afae5aac523619aa462ac28bc01d142bdc17c2215fde23c9d47e not found: ID does not exist"
Oct 30 22:16:06.655055 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:06.654968    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-2lzcz"]
Oct 30 22:16:06.660842 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:06.660800    4473 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-2lzcz"]
Oct 30 22:16:08.360028 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:08.359990    4473 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="83504be1-81ee-4a97-93e3-a61ecb2f3b2d" path="/var/lib/kubelet/pods/83504be1-81ee-4a97-93e3-a61ecb2f3b2d/volumes"
Oct 30 22:16:15.767971 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:15.767865    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-7c826c-f77fdff44-mtg87"]
Oct 30 22:16:15.768396 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:15.768094    4473 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-7c826c-f77fdff44-mtg87" podUID="021c5c2e-7d54-46be-8639-9183b8021972" containerName="maas-api" containerID="cri-o://258d12f772b06455812f87f9d3a19c43732596aafc8ba3ce377b22e7dfa8b1ed" gracePeriod=30
Oct 30 22:16:16.111758 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:16.111730    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-7c826c-f77fdff44-mtg87"
Oct 30 22:16:16.270002 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:16.267446    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/021c5c2e-7d54-46be-8639-9183b8021972-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/021c5c2e-7d54-46be-8639-9183b8021972-maas-api-tls\") pod \"021c5c2e-7d54-46be-8639-9183b8021972\" (UID: \"021c5c2e-7d54-46be-8639-9183b8021972\") "
Oct 30 22:16:16.270002 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:16.267661    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/021c5c2e-7d54-46be-8639-9183b8021972-kube-api-access-hmqqb\" (UniqueName: \"kubernetes.io/projected/021c5c2e-7d54-46be-8639-9183b8021972-kube-api-access-hmqqb\") pod \"021c5c2e-7d54-46be-8639-9183b8021972\" (UID: \"021c5c2e-7d54-46be-8639-9183b8021972\") "
Oct 30 22:16:16.270002 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:16.267797    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/021c5c2e-7d54-46be-8639-9183b8021972-metrics-certs\" (UniqueName: \"kubernetes.io/secret/021c5c2e-7d54-46be-8639-9183b8021972-metrics-certs\") pod \"021c5c2e-7d54-46be-8639-9183b8021972\" (UID: \"021c5c2e-7d54-46be-8639-9183b8021972\") "
Oct 30 22:16:16.270002 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:16.268022    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/021c5c2e-7d54-46be-8639-9183b8021972-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/021c5c2e-7d54-46be-8639-9183b8021972-openshift-service-ca\") pod \"021c5c2e-7d54-46be-8639-9183b8021972\" (UID: \"021c5c2e-7d54-46be-8639-9183b8021972\") "
Oct 30 22:16:16.270002 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:16.268393    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/021c5c2e-7d54-46be-8639-9183b8021972-openshift-service-ca" pod "021c5c2e-7d54-46be-8639-9183b8021972" (UID: "021c5c2e-7d54-46be-8639-9183b8021972"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:16:16.270002 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:16.269061    4473 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/021c5c2e-7d54-46be-8639-9183b8021972-openshift-service-ca\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:16:16.273973 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:16.272180    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/021c5c2e-7d54-46be-8639-9183b8021972-metrics-certs" pod "021c5c2e-7d54-46be-8639-9183b8021972" (UID: "021c5c2e-7d54-46be-8639-9183b8021972"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:16:16.280070 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:16.275711    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/021c5c2e-7d54-46be-8639-9183b8021972-maas-api-tls" pod "021c5c2e-7d54-46be-8639-9183b8021972" (UID: "021c5c2e-7d54-46be-8639-9183b8021972"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:16:16.285382 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:16.285159    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/021c5c2e-7d54-46be-8639-9183b8021972-kube-api-access-hmqqb" pod "021c5c2e-7d54-46be-8639-9183b8021972" (UID: "021c5c2e-7d54-46be-8639-9183b8021972"). InnerVolumeSpecName "kube-api-access-hmqqb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:16:16.399944 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:16.399649    4473 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/021c5c2e-7d54-46be-8639-9183b8021972-metrics-certs\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:16:16.399944 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:16.399686    4473 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/021c5c2e-7d54-46be-8639-9183b8021972-maas-api-tls\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:16:16.399944 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:16.399701    4473 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hmqqb\" (UniqueName: \"kubernetes.io/projected/021c5c2e-7d54-46be-8639-9183b8021972-kube-api-access-hmqqb\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:16:16.577169 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:16.577131    4473 generic.go:356] "Generic (PLEG): container finished" podID="021c5c2e-7d54-46be-8639-9183b8021972" containerID="258d12f772b06455812f87f9d3a19c43732596aafc8ba3ce377b22e7dfa8b1ed" exitCode=0
Oct 30 22:16:16.577342 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:16.577182    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-7c826c-f77fdff44-mtg87"
Oct 30 22:16:16.577342 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:16.577211    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-7c826c-f77fdff44-mtg87" event={"ID":"021c5c2e-7d54-46be-8639-9183b8021972","Type":"ContainerDied","Data":"258d12f772b06455812f87f9d3a19c43732596aafc8ba3ce377b22e7dfa8b1ed"}
Oct 30 22:16:16.577342 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:16.577265    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-7c826c-f77fdff44-mtg87" event={"ID":"021c5c2e-7d54-46be-8639-9183b8021972","Type":"ContainerDied","Data":"ab34bf4deba863523ff60a176391f997ac2246c32bdcc97218444db1a1f37259"}
Oct 30 22:16:16.577342 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:16.577286    4473 scope.go:122] "RemoveContainer" containerID="258d12f772b06455812f87f9d3a19c43732596aafc8ba3ce377b22e7dfa8b1ed"
Oct 30 22:16:16.587885 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:16.587860    4473 scope.go:122] "RemoveContainer" containerID="258d12f772b06455812f87f9d3a19c43732596aafc8ba3ce377b22e7dfa8b1ed"
Oct 30 22:16:16.588243 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:16:16.588222    4473 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"258d12f772b06455812f87f9d3a19c43732596aafc8ba3ce377b22e7dfa8b1ed\": container with ID starting with 258d12f772b06455812f87f9d3a19c43732596aafc8ba3ce377b22e7dfa8b1ed not found: ID does not exist" containerID="258d12f772b06455812f87f9d3a19c43732596aafc8ba3ce377b22e7dfa8b1ed"
Oct 30 22:16:16.588330 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:16.588252    4473 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"258d12f772b06455812f87f9d3a19c43732596aafc8ba3ce377b22e7dfa8b1ed"} err="failed to get container status \"258d12f772b06455812f87f9d3a19c43732596aafc8ba3ce377b22e7dfa8b1ed\": rpc error: code = NotFound desc = could not find container \"258d12f772b06455812f87f9d3a19c43732596aafc8ba3ce377b22e7dfa8b1ed\": container with ID starting with 258d12f772b06455812f87f9d3a19c43732596aafc8ba3ce377b22e7dfa8b1ed not found: ID does not exist"
Oct 30 22:16:16.842863 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:16.842778    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-7c826c-f77fdff44-mtg87"]
Oct 30 22:16:16.867393 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:16.867365    4473 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-7c826c-f77fdff44-mtg87"]
Oct 30 22:16:17.929150 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:17.929118    4473 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w5-5dff71-openshift-default-79bd87bcfc-jng5v"]
Oct 30 22:16:17.929871 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:17.929852    4473 cpu_manager.go:414] "containerMap: removing container" podUID="2810c233-3078-4bc0-bb3e-bc863e7786d2" containerName="payload-pre-processing"
Oct 30 22:16:17.929914 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:17.929872    4473 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2810c233-3078-4bc0-bb3e-bc863e7786d2" containerName="payload-pre-processing"
Oct 30 22:16:17.929914 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:17.929887    4473 cpu_manager.go:414] "containerMap: removing container" podUID="3e4b6a7f-8bfd-49d8-be83-674d091c99d4" containerName="istio-proxy"
Oct 30 22:16:17.929914 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:17.929892    4473 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3e4b6a7f-8bfd-49d8-be83-674d091c99d4" containerName="istio-proxy"
Oct 30 22:16:17.929914 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:17.929907    4473 cpu_manager.go:414] "containerMap: removing container" podUID="3e4b6a7f-8bfd-49d8-be83-674d091c99d4" containerName="istio-proxy"
Oct 30 22:16:17.929914 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:17.929912    4473 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3e4b6a7f-8bfd-49d8-be83-674d091c99d4" containerName="istio-proxy"
Oct 30 22:16:17.929914 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:17.929952    4473 cpu_manager.go:414] "containerMap: removing container" podUID="021c5c2e-7d54-46be-8639-9183b8021972" containerName="maas-api"
Oct 30 22:16:17.930149 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:17.929957    4473 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="021c5c2e-7d54-46be-8639-9183b8021972" containerName="maas-api"
Oct 30 22:16:17.930149 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:17.929966    4473 cpu_manager.go:414] "containerMap: removing container" podUID="13ab2734-6186-480c-ae76-6f00a295e1e1" containerName="payload-processing"
Oct 30 22:16:17.930149 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:17.929972    4473 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="13ab2734-6186-480c-ae76-6f00a295e1e1" containerName="payload-processing"
Oct 30 22:16:17.930149 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:17.929981    4473 cpu_manager.go:414] "containerMap: removing container" podUID="83504be1-81ee-4a97-93e3-a61ecb2f3b2d" containerName="istio-proxy"
Oct 30 22:16:17.930149 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:17.929985    4473 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="83504be1-81ee-4a97-93e3-a61ecb2f3b2d" containerName="istio-proxy"
Oct 30 22:16:17.930149 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:17.929999    4473 cpu_manager.go:414] "containerMap: removing container" podUID="b72da9e3-1c30-4fb9-a879-1279c67a7453" containerName="payload-processing"
Oct 30 22:16:17.930149 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:17.930003    4473 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b72da9e3-1c30-4fb9-a879-1279c67a7453" containerName="payload-processing"
Oct 30 22:16:17.930149 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:17.930011    4473 cpu_manager.go:414] "containerMap: removing container" podUID="75c6c21f-c5c0-4f9e-9f12-621e01735dda" containerName="payload-pre-processing"
Oct 30 22:16:17.930149 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:17.930016    4473 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="75c6c21f-c5c0-4f9e-9f12-621e01735dda" containerName="payload-pre-processing"
Oct 30 22:16:17.930149 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:17.930123    4473 memory_manager.go:365] "RemoveStaleState removing state" podUID="2810c233-3078-4bc0-bb3e-bc863e7786d2" containerName="payload-pre-processing"
Oct 30 22:16:17.930149 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:17.930133    4473 memory_manager.go:365] "RemoveStaleState removing state" podUID="3e4b6a7f-8bfd-49d8-be83-674d091c99d4" containerName="istio-proxy"
Oct 30 22:16:17.930149 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:17.930141    4473 memory_manager.go:365] "RemoveStaleState removing state" podUID="3e4b6a7f-8bfd-49d8-be83-674d091c99d4" containerName="istio-proxy"
Oct 30 22:16:17.930149 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:17.930149    4473 memory_manager.go:365] "RemoveStaleState removing state" podUID="021c5c2e-7d54-46be-8639-9183b8021972" containerName="maas-api"
Oct 30 22:16:17.930149 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:17.930155    4473 memory_manager.go:365] "RemoveStaleState removing state" podUID="13ab2734-6186-480c-ae76-6f00a295e1e1" containerName="payload-processing"
Oct 30 22:16:17.930585 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:17.930162    4473 memory_manager.go:365] "RemoveStaleState removing state" podUID="83504be1-81ee-4a97-93e3-a61ecb2f3b2d" containerName="istio-proxy"
Oct 30 22:16:17.930585 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:17.930168    4473 memory_manager.go:365] "RemoveStaleState removing state" podUID="b72da9e3-1c30-4fb9-a879-1279c67a7453" containerName="payload-processing"
Oct 30 22:16:17.930585 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:17.930173    4473 memory_manager.go:365] "RemoveStaleState removing state" podUID="75c6c21f-c5c0-4f9e-9f12-621e01735dda" containerName="payload-pre-processing"
Oct 30 22:16:17.935510 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:17.935486    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w5-5dff71-openshift-default-79bd87bcfc-jng5v"
Oct 30 22:16:18.041887 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:18.041857    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w5-5dff71-openshift-default-dockercfg-lfwp9\""
Oct 30 22:16:18.047281 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:18.047253    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w5-5dff71-openshift-default-79bd87bcfc-jng5v"]
Oct 30 22:16:18.136300 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:18.136256    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eadc7f9e-be79-4abe-9a9d-da311992b192-istio-token\") pod \"e2e-shared-b-w5-5dff71-openshift-default-79bd87bcfc-jng5v\" (UID: \"eadc7f9e-be79-4abe-9a9d-da311992b192\") " pod="openshift-ingress/e2e-shared-b-w5-5dff71-openshift-default-79bd87bcfc-jng5v"
Oct 30 22:16:18.136500 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:18.136329    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eadc7f9e-be79-4abe-9a9d-da311992b192-workload-certs\") pod \"e2e-shared-b-w5-5dff71-openshift-default-79bd87bcfc-jng5v\" (UID: \"eadc7f9e-be79-4abe-9a9d-da311992b192\") " pod="openshift-ingress/e2e-shared-b-w5-5dff71-openshift-default-79bd87bcfc-jng5v"
Oct 30 22:16:18.136500 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:18.136415    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eadc7f9e-be79-4abe-9a9d-da311992b192-istio-data\") pod \"e2e-shared-b-w5-5dff71-openshift-default-79bd87bcfc-jng5v\" (UID: \"eadc7f9e-be79-4abe-9a9d-da311992b192\") " pod="openshift-ingress/e2e-shared-b-w5-5dff71-openshift-default-79bd87bcfc-jng5v"
Oct 30 22:16:18.136500 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:18.136461    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eadc7f9e-be79-4abe-9a9d-da311992b192-istiod-ca-cert\") pod \"e2e-shared-b-w5-5dff71-openshift-default-79bd87bcfc-jng5v\" (UID: \"eadc7f9e-be79-4abe-9a9d-da311992b192\") " pod="openshift-ingress/e2e-shared-b-w5-5dff71-openshift-default-79bd87bcfc-jng5v"
Oct 30 22:16:18.136622 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:18.136552    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eadc7f9e-be79-4abe-9a9d-da311992b192-credential-socket\") pod \"e2e-shared-b-w5-5dff71-openshift-default-79bd87bcfc-jng5v\" (UID: \"eadc7f9e-be79-4abe-9a9d-da311992b192\") " pod="openshift-ingress/e2e-shared-b-w5-5dff71-openshift-default-79bd87bcfc-jng5v"
Oct 30 22:16:18.136708 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:18.136693    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eadc7f9e-be79-4abe-9a9d-da311992b192-istio-podinfo\") pod \"e2e-shared-b-w5-5dff71-openshift-default-79bd87bcfc-jng5v\" (UID: \"eadc7f9e-be79-4abe-9a9d-da311992b192\") " pod="openshift-ingress/e2e-shared-b-w5-5dff71-openshift-default-79bd87bcfc-jng5v"
Oct 30 22:16:18.136758 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:18.136746    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jnqmk\" (UniqueName: \"kubernetes.io/projected/eadc7f9e-be79-4abe-9a9d-da311992b192-kube-api-access-jnqmk\") pod \"e2e-shared-b-w5-5dff71-openshift-default-79bd87bcfc-jng5v\" (UID: \"eadc7f9e-be79-4abe-9a9d-da311992b192\") " pod="openshift-ingress/e2e-shared-b-w5-5dff71-openshift-default-79bd87bcfc-jng5v"
Oct 30 22:16:18.136871 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:18.136857    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eadc7f9e-be79-4abe-9a9d-da311992b192-istio-envoy\") pod \"e2e-shared-b-w5-5dff71-openshift-default-79bd87bcfc-jng5v\" (UID: \"eadc7f9e-be79-4abe-9a9d-da311992b192\") " pod="openshift-ingress/e2e-shared-b-w5-5dff71-openshift-default-79bd87bcfc-jng5v"
Oct 30 22:16:18.136908 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:18.136893    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eadc7f9e-be79-4abe-9a9d-da311992b192-workload-socket\") pod \"e2e-shared-b-w5-5dff71-openshift-default-79bd87bcfc-jng5v\" (UID: \"eadc7f9e-be79-4abe-9a9d-da311992b192\") " pod="openshift-ingress/e2e-shared-b-w5-5dff71-openshift-default-79bd87bcfc-jng5v"
Oct 30 22:16:18.237774 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:18.237684    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eadc7f9e-be79-4abe-9a9d-da311992b192-credential-socket\") pod \"e2e-shared-b-w5-5dff71-openshift-default-79bd87bcfc-jng5v\" (UID: \"eadc7f9e-be79-4abe-9a9d-da311992b192\") " pod="openshift-ingress/e2e-shared-b-w5-5dff71-openshift-default-79bd87bcfc-jng5v"
Oct 30 22:16:18.237939 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:18.237827    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eadc7f9e-be79-4abe-9a9d-da311992b192-istio-podinfo\") pod \"e2e-shared-b-w5-5dff71-openshift-default-79bd87bcfc-jng5v\" (UID: \"eadc7f9e-be79-4abe-9a9d-da311992b192\") " pod="openshift-ingress/e2e-shared-b-w5-5dff71-openshift-default-79bd87bcfc-jng5v"
Oct 30 22:16:18.237939 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:18.237885    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jnqmk\" (UniqueName: \"kubernetes.io/projected/eadc7f9e-be79-4abe-9a9d-da311992b192-kube-api-access-jnqmk\") pod \"e2e-shared-b-w5-5dff71-openshift-default-79bd87bcfc-jng5v\" (UID: \"eadc7f9e-be79-4abe-9a9d-da311992b192\") " pod="openshift-ingress/e2e-shared-b-w5-5dff71-openshift-default-79bd87bcfc-jng5v"
Oct 30 22:16:18.238062 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:18.237988    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eadc7f9e-be79-4abe-9a9d-da311992b192-istio-envoy\") pod \"e2e-shared-b-w5-5dff71-openshift-default-79bd87bcfc-jng5v\" (UID: \"eadc7f9e-be79-4abe-9a9d-da311992b192\") " pod="openshift-ingress/e2e-shared-b-w5-5dff71-openshift-default-79bd87bcfc-jng5v"
Oct 30 22:16:18.238062 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:18.238034    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eadc7f9e-be79-4abe-9a9d-da311992b192-workload-socket\") pod \"e2e-shared-b-w5-5dff71-openshift-default-79bd87bcfc-jng5v\" (UID: \"eadc7f9e-be79-4abe-9a9d-da311992b192\") " pod="openshift-ingress/e2e-shared-b-w5-5dff71-openshift-default-79bd87bcfc-jng5v"
Oct 30 22:16:18.238185 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:18.238084    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eadc7f9e-be79-4abe-9a9d-da311992b192-istio-token\") pod \"e2e-shared-b-w5-5dff71-openshift-default-79bd87bcfc-jng5v\" (UID: \"eadc7f9e-be79-4abe-9a9d-da311992b192\") " pod="openshift-ingress/e2e-shared-b-w5-5dff71-openshift-default-79bd87bcfc-jng5v"
Oct 30 22:16:18.238185 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:18.238133    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eadc7f9e-be79-4abe-9a9d-da311992b192-workload-certs\") pod \"e2e-shared-b-w5-5dff71-openshift-default-79bd87bcfc-jng5v\" (UID: \"eadc7f9e-be79-4abe-9a9d-da311992b192\") " pod="openshift-ingress/e2e-shared-b-w5-5dff71-openshift-default-79bd87bcfc-jng5v"
Oct 30 22:16:18.238185 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:18.238154    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eadc7f9e-be79-4abe-9a9d-da311992b192-istio-data\") pod \"e2e-shared-b-w5-5dff71-openshift-default-79bd87bcfc-jng5v\" (UID: \"eadc7f9e-be79-4abe-9a9d-da311992b192\") " pod="openshift-ingress/e2e-shared-b-w5-5dff71-openshift-default-79bd87bcfc-jng5v"
Oct 30 22:16:18.238185 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:18.238164    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eadc7f9e-be79-4abe-9a9d-da311992b192-credential-socket\") pod \"e2e-shared-b-w5-5dff71-openshift-default-79bd87bcfc-jng5v\" (UID: \"eadc7f9e-be79-4abe-9a9d-da311992b192\") " pod="openshift-ingress/e2e-shared-b-w5-5dff71-openshift-default-79bd87bcfc-jng5v"
Oct 30 22:16:18.238185 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:18.238178    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eadc7f9e-be79-4abe-9a9d-da311992b192-istiod-ca-cert\") pod \"e2e-shared-b-w5-5dff71-openshift-default-79bd87bcfc-jng5v\" (UID: \"eadc7f9e-be79-4abe-9a9d-da311992b192\") " pod="openshift-ingress/e2e-shared-b-w5-5dff71-openshift-default-79bd87bcfc-jng5v"
Oct 30 22:16:18.238481 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:18.238457    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eadc7f9e-be79-4abe-9a9d-da311992b192-istio-data\") pod \"e2e-shared-b-w5-5dff71-openshift-default-79bd87bcfc-jng5v\" (UID: \"eadc7f9e-be79-4abe-9a9d-da311992b192\") " pod="openshift-ingress/e2e-shared-b-w5-5dff71-openshift-default-79bd87bcfc-jng5v"
Oct 30 22:16:18.238563 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:18.238541    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eadc7f9e-be79-4abe-9a9d-da311992b192-workload-socket\") pod \"e2e-shared-b-w5-5dff71-openshift-default-79bd87bcfc-jng5v\" (UID: \"eadc7f9e-be79-4abe-9a9d-da311992b192\") " pod="openshift-ingress/e2e-shared-b-w5-5dff71-openshift-default-79bd87bcfc-jng5v"
Oct 30 22:16:18.238756 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:18.238736    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eadc7f9e-be79-4abe-9a9d-da311992b192-workload-certs\") pod \"e2e-shared-b-w5-5dff71-openshift-default-79bd87bcfc-jng5v\" (UID: \"eadc7f9e-be79-4abe-9a9d-da311992b192\") " pod="openshift-ingress/e2e-shared-b-w5-5dff71-openshift-default-79bd87bcfc-jng5v"
Oct 30 22:16:18.238858 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:18.238838    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eadc7f9e-be79-4abe-9a9d-da311992b192-istiod-ca-cert\") pod \"e2e-shared-b-w5-5dff71-openshift-default-79bd87bcfc-jng5v\" (UID: \"eadc7f9e-be79-4abe-9a9d-da311992b192\") " pod="openshift-ingress/e2e-shared-b-w5-5dff71-openshift-default-79bd87bcfc-jng5v"
Oct 30 22:16:18.240982 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:18.240960    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eadc7f9e-be79-4abe-9a9d-da311992b192-istio-envoy\") pod \"e2e-shared-b-w5-5dff71-openshift-default-79bd87bcfc-jng5v\" (UID: \"eadc7f9e-be79-4abe-9a9d-da311992b192\") " pod="openshift-ingress/e2e-shared-b-w5-5dff71-openshift-default-79bd87bcfc-jng5v"
Oct 30 22:16:18.241538 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:18.241518    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eadc7f9e-be79-4abe-9a9d-da311992b192-istio-podinfo\") pod \"e2e-shared-b-w5-5dff71-openshift-default-79bd87bcfc-jng5v\" (UID: \"eadc7f9e-be79-4abe-9a9d-da311992b192\") " pod="openshift-ingress/e2e-shared-b-w5-5dff71-openshift-default-79bd87bcfc-jng5v"
Oct 30 22:16:18.300520 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:18.300448    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eadc7f9e-be79-4abe-9a9d-da311992b192-istio-token\") pod \"e2e-shared-b-w5-5dff71-openshift-default-79bd87bcfc-jng5v\" (UID: \"eadc7f9e-be79-4abe-9a9d-da311992b192\") " pod="openshift-ingress/e2e-shared-b-w5-5dff71-openshift-default-79bd87bcfc-jng5v"
Oct 30 22:16:18.303103 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:18.303036    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jnqmk\" (UniqueName: \"kubernetes.io/projected/eadc7f9e-be79-4abe-9a9d-da311992b192-kube-api-access-jnqmk\") pod \"e2e-shared-b-w5-5dff71-openshift-default-79bd87bcfc-jng5v\" (UID: \"eadc7f9e-be79-4abe-9a9d-da311992b192\") " pod="openshift-ingress/e2e-shared-b-w5-5dff71-openshift-default-79bd87bcfc-jng5v"
Oct 30 22:16:18.358790 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:18.358760    4473 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="021c5c2e-7d54-46be-8639-9183b8021972" path="/var/lib/kubelet/pods/021c5c2e-7d54-46be-8639-9183b8021972/volumes"
Oct 30 22:16:18.552579 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:18.552485    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w5-5dff71-openshift-default-79bd87bcfc-jng5v"
Oct 30 22:16:18.988553 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:18.988519    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w5-5dff71-openshift-default-79bd87bcfc-jng5v"]
Oct 30 22:16:18.989074 ip-10-0-140-18 kubenswrapper[4473]: W1030 22:16:18.989049    4473 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeadc7f9e_be79_4abe_9a9d_da311992b192.slice/crio-343c2408310ed090c010a9e866158e2191c1999a180a4a6d8d01cddeb33870b4 WatchSource:0}: Error finding container 343c2408310ed090c010a9e866158e2191c1999a180a4a6d8d01cddeb33870b4: Status 404 returned error can't find the container with id 343c2408310ed090c010a9e866158e2191c1999a180a4a6d8d01cddeb33870b4
Oct 30 22:16:18.991958 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:18.991904    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:16:18.992057 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:18.991994    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:16:18.992057 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:18.992023    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:16:19.600799 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:19.600762    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w5-5dff71-openshift-default-79bd87bcfc-jng5v" event={"ID":"eadc7f9e-be79-4abe-9a9d-da311992b192","Type":"ContainerStarted","Data":"54375898e617206003c59fb2f48160a64a8553ca3d642e860e3cdba8bd78515f"}
Oct 30 22:16:19.601044 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:19.600828    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w5-5dff71-openshift-default-79bd87bcfc-jng5v" event={"ID":"eadc7f9e-be79-4abe-9a9d-da311992b192","Type":"ContainerStarted","Data":"343c2408310ed090c010a9e866158e2191c1999a180a4a6d8d01cddeb33870b4"}
Oct 30 22:16:19.824839 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:19.824783    4473 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w5-5dff71-openshift-default-79bd87bcfc-jng5v" podStartSLOduration=2.824768055 podStartE2EDuration="2.824768055s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:16:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:16:19.774394889 +0000 UTC m=+2025.547763376" watchObservedRunningTime="2026-10-30 22:16:19.824768055 +0000 UTC m=+2025.598136541"
Oct 30 22:16:20.354569 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:16:20.354522    4473 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fncvg" containerName="extractor"
Oct 30 22:16:20.553325 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:20.553285    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w5-5dff71-openshift-default-79bd87bcfc-jng5v"
Oct 30 22:16:20.554876 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:20.554844    4473 patch_prober.go:29] interesting pod/e2e-shared-b-w5-5dff71-openshift-default-79bd87bcfc-jng5v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.139:15021/healthz/ready\": dial tcp 10.133.0.139:15021: connect: connection refused" start-of-body=
Oct 30 22:16:20.555039 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:20.554908    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w5-5dff71-openshift-default-79bd87bcfc-jng5v" podUID="eadc7f9e-be79-4abe-9a9d-da311992b192" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.139:15021/healthz/ready\": dial tcp 10.133.0.139:15021: connect: connection refused"
Oct 30 22:16:21.553610 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:21.553570    4473 patch_prober.go:29] interesting pod/e2e-shared-b-w5-5dff71-openshift-default-79bd87bcfc-jng5v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.139:15021/healthz/ready\": dial tcp 10.133.0.139:15021: connect: connection refused" start-of-body=
Oct 30 22:16:21.554070 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:21.553654    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w5-5dff71-openshift-default-79bd87bcfc-jng5v" podUID="eadc7f9e-be79-4abe-9a9d-da311992b192" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.139:15021/healthz/ready\": dial tcp 10.133.0.139:15021: connect: connection refused"
Oct 30 22:16:22.556881 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:22.556845    4473 patch_prober.go:29] interesting pod/e2e-shared-b-w5-5dff71-openshift-default-79bd87bcfc-jng5v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.139:15021/healthz/ready\": dial tcp 10.133.0.139:15021: connect: connection refused" start-of-body=
Oct 30 22:16:22.557481 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:22.557451    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w5-5dff71-openshift-default-79bd87bcfc-jng5v" podUID="eadc7f9e-be79-4abe-9a9d-da311992b192" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.139:15021/healthz/ready\": dial tcp 10.133.0.139:15021: connect: connection refused"
Oct 30 22:16:23.553514 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:23.553462    4473 patch_prober.go:29] interesting pod/e2e-shared-b-w5-5dff71-openshift-default-79bd87bcfc-jng5v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.139:15021/healthz/ready\": dial tcp 10.133.0.139:15021: connect: connection refused" start-of-body=
Oct 30 22:16:23.553738 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:23.553529    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w5-5dff71-openshift-default-79bd87bcfc-jng5v" podUID="eadc7f9e-be79-4abe-9a9d-da311992b192" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.139:15021/healthz/ready\": dial tcp 10.133.0.139:15021: connect: connection refused"
Oct 30 22:16:24.552970 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:24.552907    4473 patch_prober.go:29] interesting pod/e2e-shared-b-w5-5dff71-openshift-default-79bd87bcfc-jng5v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.139:15021/healthz/ready\": dial tcp 10.133.0.139:15021: connect: connection refused" start-of-body=
Oct 30 22:16:24.553414 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:24.553002    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w5-5dff71-openshift-default-79bd87bcfc-jng5v" podUID="eadc7f9e-be79-4abe-9a9d-da311992b192" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.139:15021/healthz/ready\": dial tcp 10.133.0.139:15021: connect: connection refused"
Oct 30 22:16:25.584479 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:25.584444    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w5-5dff71-openshift-default-79bd87bcfc-jng5v"
Oct 30 22:16:25.584986 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:25.584960    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w5-5dff71-openshift-default-79bd87bcfc-jng5v"
Oct 30 22:16:25.585749 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:25.585728    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w5-5dff71-openshift-default-79bd87bcfc-jng5v"
Oct 30 22:16:27.868163 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:27.868129    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-5d36f6-7dc89f4558-rwzdn"]
Oct 30 22:16:27.868578 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:27.868389    4473 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-5d36f6-7dc89f4558-rwzdn" podUID="fe2360b8-5fff-43b6-bfa7-9f3e86cdbbcf" containerName="maas-api" containerID="cri-o://f57fc771af2403ebc282841fd6534b1bce7b2aeff06d77490cb0023f5b53db06" gracePeriod=30
Oct 30 22:16:28.237504 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:28.237477    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-5d36f6-7dc89f4558-rwzdn"
Oct 30 22:16:28.348518 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:28.348419    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/fe2360b8-5fff-43b6-bfa7-9f3e86cdbbcf-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fe2360b8-5fff-43b6-bfa7-9f3e86cdbbcf-maas-api-tls\") pod \"fe2360b8-5fff-43b6-bfa7-9f3e86cdbbcf\" (UID: \"fe2360b8-5fff-43b6-bfa7-9f3e86cdbbcf\") "
Oct 30 22:16:28.348788 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:28.348762    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/fe2360b8-5fff-43b6-bfa7-9f3e86cdbbcf-metrics-certs\" (UniqueName: \"kubernetes.io/secret/fe2360b8-5fff-43b6-bfa7-9f3e86cdbbcf-metrics-certs\") pod \"fe2360b8-5fff-43b6-bfa7-9f3e86cdbbcf\" (UID: \"fe2360b8-5fff-43b6-bfa7-9f3e86cdbbcf\") "
Oct 30 22:16:28.349024 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:28.349003    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fe2360b8-5fff-43b6-bfa7-9f3e86cdbbcf-kube-api-access-xf6kk\" (UniqueName: \"kubernetes.io/projected/fe2360b8-5fff-43b6-bfa7-9f3e86cdbbcf-kube-api-access-xf6kk\") pod \"fe2360b8-5fff-43b6-bfa7-9f3e86cdbbcf\" (UID: \"fe2360b8-5fff-43b6-bfa7-9f3e86cdbbcf\") "
Oct 30 22:16:28.349304 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:28.349286    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fe2360b8-5fff-43b6-bfa7-9f3e86cdbbcf-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fe2360b8-5fff-43b6-bfa7-9f3e86cdbbcf-openshift-service-ca\") pod \"fe2360b8-5fff-43b6-bfa7-9f3e86cdbbcf\" (UID: \"fe2360b8-5fff-43b6-bfa7-9f3e86cdbbcf\") "
Oct 30 22:16:28.358411 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:28.358373    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fe2360b8-5fff-43b6-bfa7-9f3e86cdbbcf-openshift-service-ca" pod "fe2360b8-5fff-43b6-bfa7-9f3e86cdbbcf" (UID: "fe2360b8-5fff-43b6-bfa7-9f3e86cdbbcf"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:16:28.359527 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:28.359501    4473 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fe2360b8-5fff-43b6-bfa7-9f3e86cdbbcf-openshift-service-ca\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:16:28.369459 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:28.369415    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fe2360b8-5fff-43b6-bfa7-9f3e86cdbbcf-maas-api-tls" pod "fe2360b8-5fff-43b6-bfa7-9f3e86cdbbcf" (UID: "fe2360b8-5fff-43b6-bfa7-9f3e86cdbbcf"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:16:28.387118 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:28.387081    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fe2360b8-5fff-43b6-bfa7-9f3e86cdbbcf-metrics-certs" pod "fe2360b8-5fff-43b6-bfa7-9f3e86cdbbcf" (UID: "fe2360b8-5fff-43b6-bfa7-9f3e86cdbbcf"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:16:28.396609 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:28.396572    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fe2360b8-5fff-43b6-bfa7-9f3e86cdbbcf-kube-api-access-xf6kk" pod "fe2360b8-5fff-43b6-bfa7-9f3e86cdbbcf" (UID: "fe2360b8-5fff-43b6-bfa7-9f3e86cdbbcf"). InnerVolumeSpecName "kube-api-access-xf6kk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:16:28.460657 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:28.460624    4473 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fe2360b8-5fff-43b6-bfa7-9f3e86cdbbcf-maas-api-tls\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:16:28.460657 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:28.460656    4473 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fe2360b8-5fff-43b6-bfa7-9f3e86cdbbcf-metrics-certs\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:16:28.460967 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:28.460669    4473 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xf6kk\" (UniqueName: \"kubernetes.io/projected/fe2360b8-5fff-43b6-bfa7-9f3e86cdbbcf-kube-api-access-xf6kk\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:16:28.689443 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:28.689414    4473 generic.go:356] "Generic (PLEG): container finished" podID="fe2360b8-5fff-43b6-bfa7-9f3e86cdbbcf" containerID="f57fc771af2403ebc282841fd6534b1bce7b2aeff06d77490cb0023f5b53db06" exitCode=0
Oct 30 22:16:28.689696 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:28.689473    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-5d36f6-7dc89f4558-rwzdn"
Oct 30 22:16:28.689696 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:28.689488    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-5d36f6-7dc89f4558-rwzdn" event={"ID":"fe2360b8-5fff-43b6-bfa7-9f3e86cdbbcf","Type":"ContainerDied","Data":"f57fc771af2403ebc282841fd6534b1bce7b2aeff06d77490cb0023f5b53db06"}
Oct 30 22:16:28.689696 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:28.689529    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-5d36f6-7dc89f4558-rwzdn" event={"ID":"fe2360b8-5fff-43b6-bfa7-9f3e86cdbbcf","Type":"ContainerDied","Data":"3d26706bdf272189880de76b65ccd84a7a044040b188febcafef34d23afba73d"}
Oct 30 22:16:28.689696 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:28.689560    4473 scope.go:122] "RemoveContainer" containerID="f57fc771af2403ebc282841fd6534b1bce7b2aeff06d77490cb0023f5b53db06"
Oct 30 22:16:28.700189 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:28.700162    4473 scope.go:122] "RemoveContainer" containerID="f57fc771af2403ebc282841fd6534b1bce7b2aeff06d77490cb0023f5b53db06"
Oct 30 22:16:28.700488 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:16:28.700469    4473 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f57fc771af2403ebc282841fd6534b1bce7b2aeff06d77490cb0023f5b53db06\": container with ID starting with f57fc771af2403ebc282841fd6534b1bce7b2aeff06d77490cb0023f5b53db06 not found: ID does not exist" containerID="f57fc771af2403ebc282841fd6534b1bce7b2aeff06d77490cb0023f5b53db06"
Oct 30 22:16:28.700543 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:28.700499    4473 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f57fc771af2403ebc282841fd6534b1bce7b2aeff06d77490cb0023f5b53db06"} err="failed to get container status \"f57fc771af2403ebc282841fd6534b1bce7b2aeff06d77490cb0023f5b53db06\": rpc error: code = NotFound desc = could not find container \"f57fc771af2403ebc282841fd6534b1bce7b2aeff06d77490cb0023f5b53db06\": container with ID starting with f57fc771af2403ebc282841fd6534b1bce7b2aeff06d77490cb0023f5b53db06 not found: ID does not exist"
Oct 30 22:16:28.858789 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:28.858751    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-5d36f6-7dc89f4558-rwzdn"]
Oct 30 22:16:28.864171 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:28.864127    4473 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-5d36f6-7dc89f4558-rwzdn"]
Oct 30 22:16:30.364995 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:30.364955    4473 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fe2360b8-5fff-43b6-bfa7-9f3e86cdbbcf" path="/var/lib/kubelet/pods/fe2360b8-5fff-43b6-bfa7-9f3e86cdbbcf/volumes"
Oct 30 22:16:30.435762 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:30.435726    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-0c99b1/distinct-2-w2-0c99b1-kserve-7466864658-8z7fk"]
Oct 30 22:16:37.541015 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:37.540931    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w2-0c99b1-64b58dc987-5h7bc"]
Oct 30 22:16:37.645237 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:37.645190    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w2-0c99b1-775568968c-tltkr"]
Oct 30 22:16:37.728423 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:37.728388    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w6-5d36f6-openshift-default-555f9495c9-9lsxw"]
Oct 30 22:16:37.728950 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:37.728904    4473 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w6-5d36f6-openshift-default-555f9495c9-9lsxw" podUID="feeeb03e-f851-4d1e-bcff-0a934226a2d1" containerName="istio-proxy" containerID="cri-o://78c8b90febdf5074794c6910486d9a38965db527e48121b57de16c57225405b6" gracePeriod=30
Oct 30 22:16:42.541460 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:42.541419    4473 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w2-0c99b1-64b58dc987-5h7bc" podUID="405c4ba4-c940-45e9-bcb9-484428bf5c5b" containerName="payload-processing" containerID="cri-o://01b67c114f1f80a3d77401e5a41c2dbea7e7cc1e1f2059a69a7093920ba28150" gracePeriod=25
Oct 30 22:16:42.646578 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:42.646531    4473 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-0c99b1-775568968c-tltkr" podUID="f260821f-0996-4717-9f4f-c30f41c238cb" containerName="payload-pre-processing" containerID="cri-o://f98299dcc16d9ec252eb58e0ef0b602f9e042b5109b121256b81450a8f29302a" gracePeriod=25
Oct 30 22:16:42.805498 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:42.805409    4473 generic.go:356] "Generic (PLEG): container finished" podID="f260821f-0996-4717-9f4f-c30f41c238cb" containerID="f98299dcc16d9ec252eb58e0ef0b602f9e042b5109b121256b81450a8f29302a" exitCode=0
Oct 30 22:16:42.805498 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:42.805478    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-0c99b1-775568968c-tltkr" event={"ID":"f260821f-0996-4717-9f4f-c30f41c238cb","Type":"ContainerDied","Data":"f98299dcc16d9ec252eb58e0ef0b602f9e042b5109b121256b81450a8f29302a"}
Oct 30 22:16:43.031432 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:43.031399    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-0c99b1-775568968c-tltkr"
Oct 30 22:16:43.077694 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:43.077663    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w6-5d36f6-openshift-default-555f9495c9-9lsxw"
Oct 30 22:16:43.110649 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:43.110610    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/feeeb03e-f851-4d1e-bcff-0a934226a2d1-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/feeeb03e-f851-4d1e-bcff-0a934226a2d1-credential-socket\") pod \"feeeb03e-f851-4d1e-bcff-0a934226a2d1\" (UID: \"feeeb03e-f851-4d1e-bcff-0a934226a2d1\") "
Oct 30 22:16:43.110844 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:43.110683    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/feeeb03e-f851-4d1e-bcff-0a934226a2d1-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/feeeb03e-f851-4d1e-bcff-0a934226a2d1-istio-data\") pod \"feeeb03e-f851-4d1e-bcff-0a934226a2d1\" (UID: \"feeeb03e-f851-4d1e-bcff-0a934226a2d1\") "
Oct 30 22:16:43.110844 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:43.110792    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/feeeb03e-f851-4d1e-bcff-0a934226a2d1-istio-token\" (UniqueName: \"kubernetes.io/projected/feeeb03e-f851-4d1e-bcff-0a934226a2d1-istio-token\") pod \"feeeb03e-f851-4d1e-bcff-0a934226a2d1\" (UID: \"feeeb03e-f851-4d1e-bcff-0a934226a2d1\") "
Oct 30 22:16:43.110844 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:43.110811    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/feeeb03e-f851-4d1e-bcff-0a934226a2d1-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/feeeb03e-f851-4d1e-bcff-0a934226a2d1-workload-certs\") pod \"feeeb03e-f851-4d1e-bcff-0a934226a2d1\" (UID: \"feeeb03e-f851-4d1e-bcff-0a934226a2d1\") "
Oct 30 22:16:43.110844 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:43.110831    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/feeeb03e-f851-4d1e-bcff-0a934226a2d1-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/feeeb03e-f851-4d1e-bcff-0a934226a2d1-istio-envoy\") pod \"feeeb03e-f851-4d1e-bcff-0a934226a2d1\" (UID: \"feeeb03e-f851-4d1e-bcff-0a934226a2d1\") "
Oct 30 22:16:43.111138 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:43.110862    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/feeeb03e-f851-4d1e-bcff-0a934226a2d1-kube-api-access-wfgb4\" (UniqueName: \"kubernetes.io/projected/feeeb03e-f851-4d1e-bcff-0a934226a2d1-kube-api-access-wfgb4\") pod \"feeeb03e-f851-4d1e-bcff-0a934226a2d1\" (UID: \"feeeb03e-f851-4d1e-bcff-0a934226a2d1\") "
Oct 30 22:16:43.111138 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:43.110900    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/feeeb03e-f851-4d1e-bcff-0a934226a2d1-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/feeeb03e-f851-4d1e-bcff-0a934226a2d1-istio-podinfo\") pod \"feeeb03e-f851-4d1e-bcff-0a934226a2d1\" (UID: \"feeeb03e-f851-4d1e-bcff-0a934226a2d1\") "
Oct 30 22:16:43.111138 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:43.110988    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/feeeb03e-f851-4d1e-bcff-0a934226a2d1-credential-socket" pod "feeeb03e-f851-4d1e-bcff-0a934226a2d1" (UID: "feeeb03e-f851-4d1e-bcff-0a934226a2d1"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:16:43.111138 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:43.111052    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/feeeb03e-f851-4d1e-bcff-0a934226a2d1-istio-data" pod "feeeb03e-f851-4d1e-bcff-0a934226a2d1" (UID: "feeeb03e-f851-4d1e-bcff-0a934226a2d1"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:16:43.111138 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:43.111069    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/feeeb03e-f851-4d1e-bcff-0a934226a2d1-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/feeeb03e-f851-4d1e-bcff-0a934226a2d1-workload-socket\") pod \"feeeb03e-f851-4d1e-bcff-0a934226a2d1\" (UID: \"feeeb03e-f851-4d1e-bcff-0a934226a2d1\") "
Oct 30 22:16:43.111412 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:43.111184    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/feeeb03e-f851-4d1e-bcff-0a934226a2d1-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/feeeb03e-f851-4d1e-bcff-0a934226a2d1-istiod-ca-cert\") pod \"feeeb03e-f851-4d1e-bcff-0a934226a2d1\" (UID: \"feeeb03e-f851-4d1e-bcff-0a934226a2d1\") "
Oct 30 22:16:43.111412 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:43.111230    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/feeeb03e-f851-4d1e-bcff-0a934226a2d1-workload-certs" pod "feeeb03e-f851-4d1e-bcff-0a934226a2d1" (UID: "feeeb03e-f851-4d1e-bcff-0a934226a2d1"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:16:43.111412 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:43.111265    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f260821f-0996-4717-9f4f-c30f41c238cb-kube-api-access-lwnbv\" (UniqueName: \"kubernetes.io/projected/f260821f-0996-4717-9f4f-c30f41c238cb-kube-api-access-lwnbv\") pod \"f260821f-0996-4717-9f4f-c30f41c238cb\" (UID: \"f260821f-0996-4717-9f4f-c30f41c238cb\") "
Oct 30 22:16:43.111412 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:43.111305    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f260821f-0996-4717-9f4f-c30f41c238cb-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f260821f-0996-4717-9f4f-c30f41c238cb-plugins-config-volume\") pod \"f260821f-0996-4717-9f4f-c30f41c238cb\" (UID: \"f260821f-0996-4717-9f4f-c30f41c238cb\") "
Oct 30 22:16:43.111412 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:43.111362    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/feeeb03e-f851-4d1e-bcff-0a934226a2d1-workload-socket" pod "feeeb03e-f851-4d1e-bcff-0a934226a2d1" (UID: "feeeb03e-f851-4d1e-bcff-0a934226a2d1"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:16:43.111700 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:43.111671    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/feeeb03e-f851-4d1e-bcff-0a934226a2d1-istiod-ca-cert" pod "feeeb03e-f851-4d1e-bcff-0a934226a2d1" (UID: "feeeb03e-f851-4d1e-bcff-0a934226a2d1"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:16:43.111957 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:43.111780    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f260821f-0996-4717-9f4f-c30f41c238cb-plugins-config-volume" pod "f260821f-0996-4717-9f4f-c30f41c238cb" (UID: "f260821f-0996-4717-9f4f-c30f41c238cb"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:16:43.112229 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:43.112205    4473 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/feeeb03e-f851-4d1e-bcff-0a934226a2d1-workload-socket\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:16:43.112305 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:43.112234    4473 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/feeeb03e-f851-4d1e-bcff-0a934226a2d1-istiod-ca-cert\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:16:43.112305 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:43.112250    4473 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f260821f-0996-4717-9f4f-c30f41c238cb-plugins-config-volume\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:16:43.112305 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:43.112266    4473 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/feeeb03e-f851-4d1e-bcff-0a934226a2d1-credential-socket\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:16:43.112305 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:43.112281    4473 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/feeeb03e-f851-4d1e-bcff-0a934226a2d1-istio-data\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:16:43.112305 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:43.112296    4473 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/feeeb03e-f851-4d1e-bcff-0a934226a2d1-workload-certs\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:16:43.113532 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:43.113504    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/feeeb03e-f851-4d1e-bcff-0a934226a2d1-istio-podinfo" pod "feeeb03e-f851-4d1e-bcff-0a934226a2d1" (UID: "feeeb03e-f851-4d1e-bcff-0a934226a2d1"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:16:43.113739 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:43.113718    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f260821f-0996-4717-9f4f-c30f41c238cb-kube-api-access-lwnbv" pod "f260821f-0996-4717-9f4f-c30f41c238cb" (UID: "f260821f-0996-4717-9f4f-c30f41c238cb"). InnerVolumeSpecName "kube-api-access-lwnbv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:16:43.113817 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:43.113776    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/feeeb03e-f851-4d1e-bcff-0a934226a2d1-kube-api-access-wfgb4" pod "feeeb03e-f851-4d1e-bcff-0a934226a2d1" (UID: "feeeb03e-f851-4d1e-bcff-0a934226a2d1"). InnerVolumeSpecName "kube-api-access-wfgb4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:16:43.113863 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:43.113849    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/feeeb03e-f851-4d1e-bcff-0a934226a2d1-istio-envoy" pod "feeeb03e-f851-4d1e-bcff-0a934226a2d1" (UID: "feeeb03e-f851-4d1e-bcff-0a934226a2d1"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:16:43.114015 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:43.113993    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/feeeb03e-f851-4d1e-bcff-0a934226a2d1-istio-token" pod "feeeb03e-f851-4d1e-bcff-0a934226a2d1" (UID: "feeeb03e-f851-4d1e-bcff-0a934226a2d1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:16:43.213375 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:43.213339    4473 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lwnbv\" (UniqueName: \"kubernetes.io/projected/f260821f-0996-4717-9f4f-c30f41c238cb-kube-api-access-lwnbv\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:16:43.213375 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:43.213368    4473 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/feeeb03e-f851-4d1e-bcff-0a934226a2d1-istio-token\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:16:43.213375 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:43.213378    4473 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/feeeb03e-f851-4d1e-bcff-0a934226a2d1-istio-envoy\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:16:43.213375 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:43.213386    4473 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wfgb4\" (UniqueName: \"kubernetes.io/projected/feeeb03e-f851-4d1e-bcff-0a934226a2d1-kube-api-access-wfgb4\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:16:43.213663 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:43.213396    4473 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/feeeb03e-f851-4d1e-bcff-0a934226a2d1-istio-podinfo\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:16:43.720426 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:43.720390    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w0-ebddcc-5f9dc96795-4z2w2"]
Oct 30 22:16:43.814467 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:43.814436    4473 generic.go:356] "Generic (PLEG): container finished" podID="feeeb03e-f851-4d1e-bcff-0a934226a2d1" containerID="78c8b90febdf5074794c6910486d9a38965db527e48121b57de16c57225405b6" exitCode=0
Oct 30 22:16:43.814644 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:43.814483    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w6-5d36f6-openshift-default-555f9495c9-9lsxw"
Oct 30 22:16:43.814644 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:43.814512    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w6-5d36f6-openshift-default-555f9495c9-9lsxw" event={"ID":"feeeb03e-f851-4d1e-bcff-0a934226a2d1","Type":"ContainerDied","Data":"78c8b90febdf5074794c6910486d9a38965db527e48121b57de16c57225405b6"}
Oct 30 22:16:43.814644 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:43.814574    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w6-5d36f6-openshift-default-555f9495c9-9lsxw" event={"ID":"feeeb03e-f851-4d1e-bcff-0a934226a2d1","Type":"ContainerDied","Data":"9314b21c4f9914d3d127c4226768cc5c56c9751221acff604745551e228235a4"}
Oct 30 22:16:43.814644 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:43.814600    4473 scope.go:122] "RemoveContainer" containerID="78c8b90febdf5074794c6910486d9a38965db527e48121b57de16c57225405b6"
Oct 30 22:16:43.816301 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:43.816265    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-0c99b1-775568968c-tltkr" event={"ID":"f260821f-0996-4717-9f4f-c30f41c238cb","Type":"ContainerDied","Data":"c7535313f67eb48cf6677a4703d9e97c47975f86618b7a8440eaf3383f7dd2fc"}
Oct 30 22:16:43.816421 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:43.816274    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-0c99b1-775568968c-tltkr"
Oct 30 22:16:43.827204 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:43.827113    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w0-ebddcc-75799c8f95-nqghx"]
Oct 30 22:16:43.828020 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:43.828000    4473 scope.go:122] "RemoveContainer" containerID="271a8613be6a58fa1d39fb6218ab6cb2dd30ddfb473d04c6bc8618966a2f20bd"
Oct 30 22:16:43.846119 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:43.846096    4473 scope.go:122] "RemoveContainer" containerID="78c8b90febdf5074794c6910486d9a38965db527e48121b57de16c57225405b6"
Oct 30 22:16:43.846457 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:16:43.846434    4473 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"78c8b90febdf5074794c6910486d9a38965db527e48121b57de16c57225405b6\": container with ID starting with 78c8b90febdf5074794c6910486d9a38965db527e48121b57de16c57225405b6 not found: ID does not exist" containerID="78c8b90febdf5074794c6910486d9a38965db527e48121b57de16c57225405b6"
Oct 30 22:16:43.846568 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:43.846469    4473 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"78c8b90febdf5074794c6910486d9a38965db527e48121b57de16c57225405b6"} err="failed to get container status \"78c8b90febdf5074794c6910486d9a38965db527e48121b57de16c57225405b6\": rpc error: code = NotFound desc = could not find container \"78c8b90febdf5074794c6910486d9a38965db527e48121b57de16c57225405b6\": container with ID starting with 78c8b90febdf5074794c6910486d9a38965db527e48121b57de16c57225405b6 not found: ID does not exist"
Oct 30 22:16:43.846568 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:43.846493    4473 scope.go:122] "RemoveContainer" containerID="271a8613be6a58fa1d39fb6218ab6cb2dd30ddfb473d04c6bc8618966a2f20bd"
Oct 30 22:16:43.846784 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:16:43.846756    4473 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"271a8613be6a58fa1d39fb6218ab6cb2dd30ddfb473d04c6bc8618966a2f20bd\": container with ID starting with 271a8613be6a58fa1d39fb6218ab6cb2dd30ddfb473d04c6bc8618966a2f20bd not found: ID does not exist" containerID="271a8613be6a58fa1d39fb6218ab6cb2dd30ddfb473d04c6bc8618966a2f20bd"
Oct 30 22:16:43.846870 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:43.846791    4473 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"271a8613be6a58fa1d39fb6218ab6cb2dd30ddfb473d04c6bc8618966a2f20bd"} err="failed to get container status \"271a8613be6a58fa1d39fb6218ab6cb2dd30ddfb473d04c6bc8618966a2f20bd\": rpc error: code = NotFound desc = could not find container \"271a8613be6a58fa1d39fb6218ab6cb2dd30ddfb473d04c6bc8618966a2f20bd\": container with ID starting with 271a8613be6a58fa1d39fb6218ab6cb2dd30ddfb473d04c6bc8618966a2f20bd not found: ID does not exist"
Oct 30 22:16:43.846870 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:43.846812    4473 scope.go:122] "RemoveContainer" containerID="f98299dcc16d9ec252eb58e0ef0b602f9e042b5109b121256b81450a8f29302a"
Oct 30 22:16:43.959469 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:43.959367    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w2-0c99b1-775568968c-tltkr"]
Oct 30 22:16:43.964685 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:43.964656    4473 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w2-0c99b1-775568968c-tltkr"]
Oct 30 22:16:44.074586 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:44.074510    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w6-5d36f6-openshift-default-555f9495c9-9lsxw"]
Oct 30 22:16:44.078914 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:44.078886    4473 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w6-5d36f6-openshift-default-555f9495c9-9lsxw"]
Oct 30 22:16:44.358882 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:44.358803    4473 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f260821f-0996-4717-9f4f-c30f41c238cb" path="/var/lib/kubelet/pods/f260821f-0996-4717-9f4f-c30f41c238cb/volumes"
Oct 30 22:16:44.359440 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:44.359424    4473 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="feeeb03e-f851-4d1e-bcff-0a934226a2d1" path="/var/lib/kubelet/pods/feeeb03e-f851-4d1e-bcff-0a934226a2d1/volumes"
Oct 30 22:16:45.458882 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:45.458837    4473 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w2-0c99b1/distinct-2-w2-0c99b1-kserve-7466864658-8z7fk" podUID="89c09555-cf53-49a7-9c3b-95d2da2a293a" containerName="main" containerID="cri-o://d9481ecfb931f5ec088cb418fe3092a6c8a8885211b0aff9d68c9157bb9e9fdc" gracePeriod=45
Oct 30 22:16:45.679946 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:45.678681    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-ch5zt"]
Oct 30 22:16:45.679946 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:45.679126    4473 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-ch5zt" podUID="7ab05096-862b-41f1-92ba-1eebceb2c026" containerName="istio-proxy" containerID="cri-o://0cb61c146d53f60d49f563776d31c3bd873a94fbc41466dc3bde1ab6ab8f130e" gracePeriod=30
Oct 30 22:16:45.864858 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:45.864821    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-0c99b1/distinct-2-w2-0c99b1-kserve-7466864658-8z7fk"
Oct 30 22:16:45.885417 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:45.885383    4473 generic.go:356] "Generic (PLEG): container finished" podID="89c09555-cf53-49a7-9c3b-95d2da2a293a" containerID="d9481ecfb931f5ec088cb418fe3092a6c8a8885211b0aff9d68c9157bb9e9fdc" exitCode=0
Oct 30 22:16:45.885597 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:45.885430    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-0c99b1/distinct-2-w2-0c99b1-kserve-7466864658-8z7fk"
Oct 30 22:16:45.885597 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:45.885452    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-0c99b1/distinct-2-w2-0c99b1-kserve-7466864658-8z7fk" event={"ID":"89c09555-cf53-49a7-9c3b-95d2da2a293a","Type":"ContainerDied","Data":"d9481ecfb931f5ec088cb418fe3092a6c8a8885211b0aff9d68c9157bb9e9fdc"}
Oct 30 22:16:45.885597 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:45.885492    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-0c99b1/distinct-2-w2-0c99b1-kserve-7466864658-8z7fk" event={"ID":"89c09555-cf53-49a7-9c3b-95d2da2a293a","Type":"ContainerDied","Data":"676a7e4eb21d8f6a63c5185e7f48c98d9faa48df88307c66ad83aef25ce405bf"}
Oct 30 22:16:45.885597 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:45.885515    4473 scope.go:122] "RemoveContainer" containerID="d9481ecfb931f5ec088cb418fe3092a6c8a8885211b0aff9d68c9157bb9e9fdc"
Oct 30 22:16:45.896707 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:45.896678    4473 scope.go:122] "RemoveContainer" containerID="d9481ecfb931f5ec088cb418fe3092a6c8a8885211b0aff9d68c9157bb9e9fdc"
Oct 30 22:16:45.897088 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:16:45.897063    4473 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d9481ecfb931f5ec088cb418fe3092a6c8a8885211b0aff9d68c9157bb9e9fdc\": container with ID starting with d9481ecfb931f5ec088cb418fe3092a6c8a8885211b0aff9d68c9157bb9e9fdc not found: ID does not exist" containerID="d9481ecfb931f5ec088cb418fe3092a6c8a8885211b0aff9d68c9157bb9e9fdc"
Oct 30 22:16:45.897176 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:45.897101    4473 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d9481ecfb931f5ec088cb418fe3092a6c8a8885211b0aff9d68c9157bb9e9fdc"} err="failed to get container status \"d9481ecfb931f5ec088cb418fe3092a6c8a8885211b0aff9d68c9157bb9e9fdc\": rpc error: code = NotFound desc = could not find container \"d9481ecfb931f5ec088cb418fe3092a6c8a8885211b0aff9d68c9157bb9e9fdc\": container with ID starting with d9481ecfb931f5ec088cb418fe3092a6c8a8885211b0aff9d68c9157bb9e9fdc not found: ID does not exist"
Oct 30 22:16:45.984042 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:45.984011    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/89c09555-cf53-49a7-9c3b-95d2da2a293a-home\" (UniqueName: \"kubernetes.io/empty-dir/89c09555-cf53-49a7-9c3b-95d2da2a293a-home\") pod \"89c09555-cf53-49a7-9c3b-95d2da2a293a\" (UID: \"89c09555-cf53-49a7-9c3b-95d2da2a293a\") "
Oct 30 22:16:45.984260 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:45.984122    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/89c09555-cf53-49a7-9c3b-95d2da2a293a-kube-api-access-qxqhm\" (UniqueName: \"kubernetes.io/projected/89c09555-cf53-49a7-9c3b-95d2da2a293a-kube-api-access-qxqhm\") pod \"89c09555-cf53-49a7-9c3b-95d2da2a293a\" (UID: \"89c09555-cf53-49a7-9c3b-95d2da2a293a\") "
Oct 30 22:16:45.984260 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:45.984190    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/89c09555-cf53-49a7-9c3b-95d2da2a293a-tls-certs\" (UniqueName: \"kubernetes.io/secret/89c09555-cf53-49a7-9c3b-95d2da2a293a-tls-certs\") pod \"89c09555-cf53-49a7-9c3b-95d2da2a293a\" (UID: \"89c09555-cf53-49a7-9c3b-95d2da2a293a\") "
Oct 30 22:16:45.984260 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:45.984247    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/89c09555-cf53-49a7-9c3b-95d2da2a293a-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/89c09555-cf53-49a7-9c3b-95d2da2a293a-tmp-dir\") pod \"89c09555-cf53-49a7-9c3b-95d2da2a293a\" (UID: \"89c09555-cf53-49a7-9c3b-95d2da2a293a\") "
Oct 30 22:16:45.984435 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:45.984318    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/89c09555-cf53-49a7-9c3b-95d2da2a293a-dshm\" (UniqueName: \"kubernetes.io/empty-dir/89c09555-cf53-49a7-9c3b-95d2da2a293a-dshm\") pod \"89c09555-cf53-49a7-9c3b-95d2da2a293a\" (UID: \"89c09555-cf53-49a7-9c3b-95d2da2a293a\") "
Oct 30 22:16:45.984544 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:45.984528    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/89c09555-cf53-49a7-9c3b-95d2da2a293a-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/89c09555-cf53-49a7-9c3b-95d2da2a293a-model-cache\") pod \"89c09555-cf53-49a7-9c3b-95d2da2a293a\" (UID: \"89c09555-cf53-49a7-9c3b-95d2da2a293a\") "
Oct 30 22:16:45.985705 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:45.985465    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/89c09555-cf53-49a7-9c3b-95d2da2a293a-tmp-dir" pod "89c09555-cf53-49a7-9c3b-95d2da2a293a" (UID: "89c09555-cf53-49a7-9c3b-95d2da2a293a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:16:45.986881 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:45.986805    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/89c09555-cf53-49a7-9c3b-95d2da2a293a-home" pod "89c09555-cf53-49a7-9c3b-95d2da2a293a" (UID: "89c09555-cf53-49a7-9c3b-95d2da2a293a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:16:45.987050 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:45.987015    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/89c09555-cf53-49a7-9c3b-95d2da2a293a-model-cache" pod "89c09555-cf53-49a7-9c3b-95d2da2a293a" (UID: "89c09555-cf53-49a7-9c3b-95d2da2a293a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:16:45.994983 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:45.991043    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/89c09555-cf53-49a7-9c3b-95d2da2a293a-tls-certs" pod "89c09555-cf53-49a7-9c3b-95d2da2a293a" (UID: "89c09555-cf53-49a7-9c3b-95d2da2a293a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:16:45.994983 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:45.992117    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/89c09555-cf53-49a7-9c3b-95d2da2a293a-kube-api-access-qxqhm" pod "89c09555-cf53-49a7-9c3b-95d2da2a293a" (UID: "89c09555-cf53-49a7-9c3b-95d2da2a293a"). InnerVolumeSpecName "kube-api-access-qxqhm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:16:45.994983 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:45.994005    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/89c09555-cf53-49a7-9c3b-95d2da2a293a-dshm" pod "89c09555-cf53-49a7-9c3b-95d2da2a293a" (UID: "89c09555-cf53-49a7-9c3b-95d2da2a293a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:16:46.086713 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:46.086671    4473 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/89c09555-cf53-49a7-9c3b-95d2da2a293a-model-cache\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:16:46.086713 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:46.086713    4473 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/89c09555-cf53-49a7-9c3b-95d2da2a293a-home\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:16:46.086713 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:46.086727    4473 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qxqhm\" (UniqueName: \"kubernetes.io/projected/89c09555-cf53-49a7-9c3b-95d2da2a293a-kube-api-access-qxqhm\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:16:46.086713 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:46.086739    4473 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/89c09555-cf53-49a7-9c3b-95d2da2a293a-tls-certs\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:16:46.086713 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:46.086751    4473 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/89c09555-cf53-49a7-9c3b-95d2da2a293a-tmp-dir\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:16:46.086713 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:46.086761    4473 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/89c09555-cf53-49a7-9c3b-95d2da2a293a-dshm\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:16:46.311966 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:46.311832    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-0c99b1/distinct-2-w2-0c99b1-kserve-7466864658-8z7fk"]
Oct 30 22:16:46.315342 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:46.315311    4473 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w2-0c99b1/distinct-2-w2-0c99b1-kserve-7466864658-8z7fk"]
Oct 30 22:16:46.360323 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:46.360294    4473 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="89c09555-cf53-49a7-9c3b-95d2da2a293a" path="/var/lib/kubelet/pods/89c09555-cf53-49a7-9c3b-95d2da2a293a/volumes"
Oct 30 22:16:47.213975 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:47.213893    4473 patch_prober.go:29] interesting pod/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-ch5zt container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 22:16:47.214370 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:47.214007    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-ch5zt" podUID="7ab05096-862b-41f1-92ba-1eebceb2c026" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 22:16:47.399810 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:47.399770    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-dd67d"]
Oct 30 22:16:47.400039 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:47.399994    4473 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-dd67d" podUID="9dc3b95a-13ab-453d-a5a5-d9037ebcfada" containerName="istio-proxy" containerID="cri-o://e932b7049a0f917709c2793d151e7e4e8cce450cf4c793d2379ae3c806f88391" gracePeriod=30
Oct 30 22:16:48.720690 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:48.720644    4473 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w0-ebddcc-5f9dc96795-4z2w2" podUID="e94e5f73-3534-41c8-aa76-38ff540bf455" containerName="payload-processing" containerID="cri-o://0ce341f1a36c5f164ab88157d590fd2a7cfd06f28c3282f617237cb18b7869d6" gracePeriod=25
Oct 30 22:16:48.827467 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:48.827414    4473 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-ebddcc-75799c8f95-nqghx" podUID="2102f3d2-e20b-402d-81ec-d6b4923df356" containerName="payload-pre-processing" containerID="cri-o://ec19e2dfd41a90a10f6da794402cdf55b1441f505fc77ac00500f3f7320e2dad" gracePeriod=25
Oct 30 22:16:48.915150 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:48.915120    4473 generic.go:356] "Generic (PLEG): container finished" podID="e94e5f73-3534-41c8-aa76-38ff540bf455" containerID="0ce341f1a36c5f164ab88157d590fd2a7cfd06f28c3282f617237cb18b7869d6" exitCode=0
Oct 30 22:16:48.915565 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:48.915371    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w0-ebddcc-5f9dc96795-4z2w2" event={"ID":"e94e5f73-3534-41c8-aa76-38ff540bf455","Type":"ContainerDied","Data":"0ce341f1a36c5f164ab88157d590fd2a7cfd06f28c3282f617237cb18b7869d6"}
Oct 30 22:16:49.015440 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:49.015410    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w0-ebddcc-5f9dc96795-4z2w2"
Oct 30 22:16:49.122784 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:49.122754    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-ebddcc-75799c8f95-nqghx"
Oct 30 22:16:49.140283 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:49.139902    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e94e5f73-3534-41c8-aa76-38ff540bf455-kube-api-access-l8dgc\" (UniqueName: \"kubernetes.io/projected/e94e5f73-3534-41c8-aa76-38ff540bf455-kube-api-access-l8dgc\") pod \"e94e5f73-3534-41c8-aa76-38ff540bf455\" (UID: \"e94e5f73-3534-41c8-aa76-38ff540bf455\") "
Oct 30 22:16:49.140283 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:49.140064    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e94e5f73-3534-41c8-aa76-38ff540bf455-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e94e5f73-3534-41c8-aa76-38ff540bf455-plugins-config-volume\") pod \"e94e5f73-3534-41c8-aa76-38ff540bf455\" (UID: \"e94e5f73-3534-41c8-aa76-38ff540bf455\") "
Oct 30 22:16:49.141473 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:49.141440    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e94e5f73-3534-41c8-aa76-38ff540bf455-plugins-config-volume" pod "e94e5f73-3534-41c8-aa76-38ff540bf455" (UID: "e94e5f73-3534-41c8-aa76-38ff540bf455"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:16:49.175353 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:49.175294    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e94e5f73-3534-41c8-aa76-38ff540bf455-kube-api-access-l8dgc" pod "e94e5f73-3534-41c8-aa76-38ff540bf455" (UID: "e94e5f73-3534-41c8-aa76-38ff540bf455"). InnerVolumeSpecName "kube-api-access-l8dgc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:16:49.242118 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:49.242022    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2102f3d2-e20b-402d-81ec-d6b4923df356-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2102f3d2-e20b-402d-81ec-d6b4923df356-plugins-config-volume\") pod \"2102f3d2-e20b-402d-81ec-d6b4923df356\" (UID: \"2102f3d2-e20b-402d-81ec-d6b4923df356\") "
Oct 30 22:16:49.242367 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:49.242216    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2102f3d2-e20b-402d-81ec-d6b4923df356-kube-api-access-jm9s6\" (UniqueName: \"kubernetes.io/projected/2102f3d2-e20b-402d-81ec-d6b4923df356-kube-api-access-jm9s6\") pod \"2102f3d2-e20b-402d-81ec-d6b4923df356\" (UID: \"2102f3d2-e20b-402d-81ec-d6b4923df356\") "
Oct 30 22:16:49.242411 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:49.242375    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2102f3d2-e20b-402d-81ec-d6b4923df356-plugins-config-volume" pod "2102f3d2-e20b-402d-81ec-d6b4923df356" (UID: "2102f3d2-e20b-402d-81ec-d6b4923df356"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:16:49.242739 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:49.242723    4473 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e94e5f73-3534-41c8-aa76-38ff540bf455-plugins-config-volume\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:16:49.242778 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:49.242744    4473 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2102f3d2-e20b-402d-81ec-d6b4923df356-plugins-config-volume\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:16:49.242778 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:49.242754    4473 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l8dgc\" (UniqueName: \"kubernetes.io/projected/e94e5f73-3534-41c8-aa76-38ff540bf455-kube-api-access-l8dgc\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:16:49.244628 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:49.244604    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2102f3d2-e20b-402d-81ec-d6b4923df356-kube-api-access-jm9s6" pod "2102f3d2-e20b-402d-81ec-d6b4923df356" (UID: "2102f3d2-e20b-402d-81ec-d6b4923df356"). InnerVolumeSpecName "kube-api-access-jm9s6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:16:49.343813 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:49.343768    4473 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jm9s6\" (UniqueName: \"kubernetes.io/projected/2102f3d2-e20b-402d-81ec-d6b4923df356-kube-api-access-jm9s6\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:16:49.924389 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:49.924348    4473 generic.go:356] "Generic (PLEG): container finished" podID="2102f3d2-e20b-402d-81ec-d6b4923df356" containerID="ec19e2dfd41a90a10f6da794402cdf55b1441f505fc77ac00500f3f7320e2dad" exitCode=0
Oct 30 22:16:49.924889 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:49.924436    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-ebddcc-75799c8f95-nqghx"
Oct 30 22:16:49.924889 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:49.924439    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-ebddcc-75799c8f95-nqghx" event={"ID":"2102f3d2-e20b-402d-81ec-d6b4923df356","Type":"ContainerDied","Data":"ec19e2dfd41a90a10f6da794402cdf55b1441f505fc77ac00500f3f7320e2dad"}
Oct 30 22:16:49.924889 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:49.924588    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-ebddcc-75799c8f95-nqghx" event={"ID":"2102f3d2-e20b-402d-81ec-d6b4923df356","Type":"ContainerDied","Data":"c049d994d5866d01bafedf36d425e6547f8024a13f8e0ca059e9239597d34e77"}
Oct 30 22:16:49.924889 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:49.924631    4473 scope.go:122] "RemoveContainer" containerID="ec19e2dfd41a90a10f6da794402cdf55b1441f505fc77ac00500f3f7320e2dad"
Oct 30 22:16:49.926329 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:49.926310    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w0-ebddcc-5f9dc96795-4z2w2"
Oct 30 22:16:49.926448 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:49.926340    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w0-ebddcc-5f9dc96795-4z2w2" event={"ID":"e94e5f73-3534-41c8-aa76-38ff540bf455","Type":"ContainerDied","Data":"af5a7a84af836932b7143554c2b2f615fea27b237cf31ace9729afaac22f9cf6"}
Oct 30 22:16:49.936076 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:49.935999    4473 scope.go:122] "RemoveContainer" containerID="ec19e2dfd41a90a10f6da794402cdf55b1441f505fc77ac00500f3f7320e2dad"
Oct 30 22:16:49.936407 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:16:49.936377    4473 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ec19e2dfd41a90a10f6da794402cdf55b1441f505fc77ac00500f3f7320e2dad\": container with ID starting with ec19e2dfd41a90a10f6da794402cdf55b1441f505fc77ac00500f3f7320e2dad not found: ID does not exist" containerID="ec19e2dfd41a90a10f6da794402cdf55b1441f505fc77ac00500f3f7320e2dad"
Oct 30 22:16:49.936508 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:49.936423    4473 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ec19e2dfd41a90a10f6da794402cdf55b1441f505fc77ac00500f3f7320e2dad"} err="failed to get container status \"ec19e2dfd41a90a10f6da794402cdf55b1441f505fc77ac00500f3f7320e2dad\": rpc error: code = NotFound desc = could not find container \"ec19e2dfd41a90a10f6da794402cdf55b1441f505fc77ac00500f3f7320e2dad\": container with ID starting with ec19e2dfd41a90a10f6da794402cdf55b1441f505fc77ac00500f3f7320e2dad not found: ID does not exist"
Oct 30 22:16:49.936508 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:49.936446    4473 scope.go:122] "RemoveContainer" containerID="0ce341f1a36c5f164ab88157d590fd2a7cfd06f28c3282f617237cb18b7869d6"
Oct 30 22:16:50.091465 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:50.091365    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w0-ebddcc-75799c8f95-nqghx"]
Oct 30 22:16:50.142317 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:50.142277    4473 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w0-ebddcc-75799c8f95-nqghx"]
Oct 30 22:16:50.252960 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:50.252804    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w0-ebddcc-5f9dc96795-4z2w2"]
Oct 30 22:16:50.256456 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:50.256407    4473 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w0-ebddcc-5f9dc96795-4z2w2"]
Oct 30 22:16:50.327127 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:16:50.327089    4473 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fncvg" containerName="extractor"
Oct 30 22:16:50.358726 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:50.358649    4473 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2102f3d2-e20b-402d-81ec-d6b4923df356" path="/var/lib/kubelet/pods/2102f3d2-e20b-402d-81ec-d6b4923df356/volumes"
Oct 30 22:16:50.359095 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:50.359079    4473 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e94e5f73-3534-41c8-aa76-38ff540bf455" path="/var/lib/kubelet/pods/e94e5f73-3534-41c8-aa76-38ff540bf455/volumes"
Oct 30 22:16:50.942455 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:50.942302    4473 generic.go:356] "Generic (PLEG): container finished" podID="7ab05096-862b-41f1-92ba-1eebceb2c026" containerID="0cb61c146d53f60d49f563776d31c3bd873a94fbc41466dc3bde1ab6ab8f130e" exitCode=0
Oct 30 22:16:50.942455 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:50.942392    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-ch5zt" event={"ID":"7ab05096-862b-41f1-92ba-1eebceb2c026","Type":"ContainerDied","Data":"0cb61c146d53f60d49f563776d31c3bd873a94fbc41466dc3bde1ab6ab8f130e"}
Oct 30 22:16:50.964808 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:50.964778    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w6-0a7679-69cc484995-hlw4x"]
Oct 30 22:16:51.002393 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:51.002365    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-ch5zt"
Oct 30 22:16:51.172156 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:51.172117    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7ab05096-862b-41f1-92ba-1eebceb2c026-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7ab05096-862b-41f1-92ba-1eebceb2c026-workload-certs\") pod \"7ab05096-862b-41f1-92ba-1eebceb2c026\" (UID: \"7ab05096-862b-41f1-92ba-1eebceb2c026\") "
Oct 30 22:16:51.172356 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:51.172268    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7ab05096-862b-41f1-92ba-1eebceb2c026-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7ab05096-862b-41f1-92ba-1eebceb2c026-istio-envoy\") pod \"7ab05096-862b-41f1-92ba-1eebceb2c026\" (UID: \"7ab05096-862b-41f1-92ba-1eebceb2c026\") "
Oct 30 22:16:51.172356 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:51.172296    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7ab05096-862b-41f1-92ba-1eebceb2c026-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7ab05096-862b-41f1-92ba-1eebceb2c026-istio-data\") pod \"7ab05096-862b-41f1-92ba-1eebceb2c026\" (UID: \"7ab05096-862b-41f1-92ba-1eebceb2c026\") "
Oct 30 22:16:51.172497 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:51.172396    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7ab05096-862b-41f1-92ba-1eebceb2c026-workload-certs" pod "7ab05096-862b-41f1-92ba-1eebceb2c026" (UID: "7ab05096-862b-41f1-92ba-1eebceb2c026"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:16:51.172497 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:51.172476    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7ab05096-862b-41f1-92ba-1eebceb2c026-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7ab05096-862b-41f1-92ba-1eebceb2c026-istiod-ca-cert\") pod \"7ab05096-862b-41f1-92ba-1eebceb2c026\" (UID: \"7ab05096-862b-41f1-92ba-1eebceb2c026\") "
Oct 30 22:16:51.172612 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:51.172528    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7ab05096-862b-41f1-92ba-1eebceb2c026-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ab05096-862b-41f1-92ba-1eebceb2c026-workload-socket\") pod \"7ab05096-862b-41f1-92ba-1eebceb2c026\" (UID: \"7ab05096-862b-41f1-92ba-1eebceb2c026\") "
Oct 30 22:16:51.172612 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:51.172579    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/7ab05096-862b-41f1-92ba-1eebceb2c026-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7ab05096-862b-41f1-92ba-1eebceb2c026-istio-podinfo\") pod \"7ab05096-862b-41f1-92ba-1eebceb2c026\" (UID: \"7ab05096-862b-41f1-92ba-1eebceb2c026\") "
Oct 30 22:16:51.172738 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:51.172717    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7ab05096-862b-41f1-92ba-1eebceb2c026-istio-token\" (UniqueName: \"kubernetes.io/projected/7ab05096-862b-41f1-92ba-1eebceb2c026-istio-token\") pod \"7ab05096-862b-41f1-92ba-1eebceb2c026\" (UID: \"7ab05096-862b-41f1-92ba-1eebceb2c026\") "
Oct 30 22:16:51.172799 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:51.172759    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7ab05096-862b-41f1-92ba-1eebceb2c026-istio-data" pod "7ab05096-862b-41f1-92ba-1eebceb2c026" (UID: "7ab05096-862b-41f1-92ba-1eebceb2c026"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:16:51.172860 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:51.172807    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7ab05096-862b-41f1-92ba-1eebceb2c026-kube-api-access-qnzv4\" (UniqueName: \"kubernetes.io/projected/7ab05096-862b-41f1-92ba-1eebceb2c026-kube-api-access-qnzv4\") pod \"7ab05096-862b-41f1-92ba-1eebceb2c026\" (UID: \"7ab05096-862b-41f1-92ba-1eebceb2c026\") "
Oct 30 22:16:51.172935 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:51.172856    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7ab05096-862b-41f1-92ba-1eebceb2c026-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ab05096-862b-41f1-92ba-1eebceb2c026-credential-socket\") pod \"7ab05096-862b-41f1-92ba-1eebceb2c026\" (UID: \"7ab05096-862b-41f1-92ba-1eebceb2c026\") "
Oct 30 22:16:51.173011 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:51.172951    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7ab05096-862b-41f1-92ba-1eebceb2c026-workload-socket" pod "7ab05096-862b-41f1-92ba-1eebceb2c026" (UID: "7ab05096-862b-41f1-92ba-1eebceb2c026"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:16:51.173258 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:51.173238    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7ab05096-862b-41f1-92ba-1eebceb2c026-istiod-ca-cert" pod "7ab05096-862b-41f1-92ba-1eebceb2c026" (UID: "7ab05096-862b-41f1-92ba-1eebceb2c026"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:16:51.173594 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:51.173571    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7ab05096-862b-41f1-92ba-1eebceb2c026-credential-socket" pod "7ab05096-862b-41f1-92ba-1eebceb2c026" (UID: "7ab05096-862b-41f1-92ba-1eebceb2c026"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:16:51.173594 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:51.173492    4473 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7ab05096-862b-41f1-92ba-1eebceb2c026-workload-certs\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:16:51.173751 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:51.173613    4473 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7ab05096-862b-41f1-92ba-1eebceb2c026-istio-data\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:16:51.173751 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:51.173628    4473 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7ab05096-862b-41f1-92ba-1eebceb2c026-istiod-ca-cert\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:16:51.173751 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:51.173642    4473 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ab05096-862b-41f1-92ba-1eebceb2c026-workload-socket\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:16:51.175191 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:51.175153    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7ab05096-862b-41f1-92ba-1eebceb2c026-istio-envoy" pod "7ab05096-862b-41f1-92ba-1eebceb2c026" (UID: "7ab05096-862b-41f1-92ba-1eebceb2c026"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:16:51.179539 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:51.179483    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7ab05096-862b-41f1-92ba-1eebceb2c026-istio-token" pod "7ab05096-862b-41f1-92ba-1eebceb2c026" (UID: "7ab05096-862b-41f1-92ba-1eebceb2c026"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:16:51.181629 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:51.181604    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7ab05096-862b-41f1-92ba-1eebceb2c026-istio-podinfo" pod "7ab05096-862b-41f1-92ba-1eebceb2c026" (UID: "7ab05096-862b-41f1-92ba-1eebceb2c026"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:16:51.181741 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:51.181628    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7ab05096-862b-41f1-92ba-1eebceb2c026-kube-api-access-qnzv4" pod "7ab05096-862b-41f1-92ba-1eebceb2c026" (UID: "7ab05096-862b-41f1-92ba-1eebceb2c026"). InnerVolumeSpecName "kube-api-access-qnzv4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:16:51.275134 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:51.275048    4473 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7ab05096-862b-41f1-92ba-1eebceb2c026-istio-token\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:16:51.275134 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:51.275083    4473 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qnzv4\" (UniqueName: \"kubernetes.io/projected/7ab05096-862b-41f1-92ba-1eebceb2c026-kube-api-access-qnzv4\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:16:51.275134 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:51.275099    4473 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ab05096-862b-41f1-92ba-1eebceb2c026-credential-socket\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:16:51.275134 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:51.275112    4473 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7ab05096-862b-41f1-92ba-1eebceb2c026-istio-envoy\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:16:51.275134 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:51.275124    4473 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7ab05096-862b-41f1-92ba-1eebceb2c026-istio-podinfo\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:16:51.451656 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:51.451616    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w2-0c99b1-64b58dc987-5h7bc" podUID="405c4ba4-c940-45e9-bcb9-484428bf5c5b" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:16:51.952536 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:51.952498    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-ch5zt"
Oct 30 22:16:51.952536 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:51.952511    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-ch5zt" event={"ID":"7ab05096-862b-41f1-92ba-1eebceb2c026","Type":"ContainerDied","Data":"e1a9c90b5bdec133574d00947aa27eb3f87dcb0edb3f5f6f1db36ba0397a703b"}
Oct 30 22:16:51.953142 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:51.952586    4473 scope.go:122] "RemoveContainer" containerID="0cb61c146d53f60d49f563776d31c3bd873a94fbc41466dc3bde1ab6ab8f130e"
Oct 30 22:16:52.169957 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:52.167639    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-ch5zt"]
Oct 30 22:16:52.203851 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:52.203762    4473 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-ch5zt"]
Oct 30 22:16:52.396641 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:52.396601    4473 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7ab05096-862b-41f1-92ba-1eebceb2c026" path="/var/lib/kubelet/pods/7ab05096-862b-41f1-92ba-1eebceb2c026/volumes"
Oct 30 22:16:52.787125 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:52.787090    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-dd67d"
Oct 30 22:16:52.805278 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:52.805242    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9dc3b95a-13ab-453d-a5a5-d9037ebcfada-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9dc3b95a-13ab-453d-a5a5-d9037ebcfada-workload-certs\") pod \"9dc3b95a-13ab-453d-a5a5-d9037ebcfada\" (UID: \"9dc3b95a-13ab-453d-a5a5-d9037ebcfada\") "
Oct 30 22:16:52.805458 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:52.805304    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9dc3b95a-13ab-453d-a5a5-d9037ebcfada-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9dc3b95a-13ab-453d-a5a5-d9037ebcfada-istio-data\") pod \"9dc3b95a-13ab-453d-a5a5-d9037ebcfada\" (UID: \"9dc3b95a-13ab-453d-a5a5-d9037ebcfada\") "
Oct 30 22:16:52.805458 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:52.805345    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9dc3b95a-13ab-453d-a5a5-d9037ebcfada-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9dc3b95a-13ab-453d-a5a5-d9037ebcfada-istiod-ca-cert\") pod \"9dc3b95a-13ab-453d-a5a5-d9037ebcfada\" (UID: \"9dc3b95a-13ab-453d-a5a5-d9037ebcfada\") "
Oct 30 22:16:52.805458 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:52.805377    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9dc3b95a-13ab-453d-a5a5-d9037ebcfada-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9dc3b95a-13ab-453d-a5a5-d9037ebcfada-istio-envoy\") pod \"9dc3b95a-13ab-453d-a5a5-d9037ebcfada\" (UID: \"9dc3b95a-13ab-453d-a5a5-d9037ebcfada\") "
Oct 30 22:16:52.805458 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:52.805403    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9dc3b95a-13ab-453d-a5a5-d9037ebcfada-kube-api-access-dxjxj\" (UniqueName: \"kubernetes.io/projected/9dc3b95a-13ab-453d-a5a5-d9037ebcfada-kube-api-access-dxjxj\") pod \"9dc3b95a-13ab-453d-a5a5-d9037ebcfada\" (UID: \"9dc3b95a-13ab-453d-a5a5-d9037ebcfada\") "
Oct 30 22:16:52.805663 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:52.805481    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9dc3b95a-13ab-453d-a5a5-d9037ebcfada-istio-token\" (UniqueName: \"kubernetes.io/projected/9dc3b95a-13ab-453d-a5a5-d9037ebcfada-istio-token\") pod \"9dc3b95a-13ab-453d-a5a5-d9037ebcfada\" (UID: \"9dc3b95a-13ab-453d-a5a5-d9037ebcfada\") "
Oct 30 22:16:52.805663 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:52.805555    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/9dc3b95a-13ab-453d-a5a5-d9037ebcfada-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9dc3b95a-13ab-453d-a5a5-d9037ebcfada-istio-podinfo\") pod \"9dc3b95a-13ab-453d-a5a5-d9037ebcfada\" (UID: \"9dc3b95a-13ab-453d-a5a5-d9037ebcfada\") "
Oct 30 22:16:52.805663 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:52.805583    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9dc3b95a-13ab-453d-a5a5-d9037ebcfada-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9dc3b95a-13ab-453d-a5a5-d9037ebcfada-workload-socket\") pod \"9dc3b95a-13ab-453d-a5a5-d9037ebcfada\" (UID: \"9dc3b95a-13ab-453d-a5a5-d9037ebcfada\") "
Oct 30 22:16:52.805663 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:52.805584    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9dc3b95a-13ab-453d-a5a5-d9037ebcfada-workload-certs" pod "9dc3b95a-13ab-453d-a5a5-d9037ebcfada" (UID: "9dc3b95a-13ab-453d-a5a5-d9037ebcfada"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:16:52.805663 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:52.805600    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9dc3b95a-13ab-453d-a5a5-d9037ebcfada-istio-data" pod "9dc3b95a-13ab-453d-a5a5-d9037ebcfada" (UID: "9dc3b95a-13ab-453d-a5a5-d9037ebcfada"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:16:52.805663 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:52.805610    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9dc3b95a-13ab-453d-a5a5-d9037ebcfada-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9dc3b95a-13ab-453d-a5a5-d9037ebcfada-credential-socket\") pod \"9dc3b95a-13ab-453d-a5a5-d9037ebcfada\" (UID: \"9dc3b95a-13ab-453d-a5a5-d9037ebcfada\") "
Oct 30 22:16:52.806044 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:52.805880    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9dc3b95a-13ab-453d-a5a5-d9037ebcfada-workload-socket" pod "9dc3b95a-13ab-453d-a5a5-d9037ebcfada" (UID: "9dc3b95a-13ab-453d-a5a5-d9037ebcfada"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:16:52.806044 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:52.805961    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9dc3b95a-13ab-453d-a5a5-d9037ebcfada-istiod-ca-cert" pod "9dc3b95a-13ab-453d-a5a5-d9037ebcfada" (UID: "9dc3b95a-13ab-453d-a5a5-d9037ebcfada"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:16:52.806164 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:52.806112    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9dc3b95a-13ab-453d-a5a5-d9037ebcfada-credential-socket" pod "9dc3b95a-13ab-453d-a5a5-d9037ebcfada" (UID: "9dc3b95a-13ab-453d-a5a5-d9037ebcfada"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:16:52.806557 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:52.806435    4473 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9dc3b95a-13ab-453d-a5a5-d9037ebcfada-workload-certs\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:16:52.806557 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:52.806464    4473 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9dc3b95a-13ab-453d-a5a5-d9037ebcfada-istio-data\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:16:52.806557 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:52.806481    4473 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9dc3b95a-13ab-453d-a5a5-d9037ebcfada-istiod-ca-cert\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:16:52.806782 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:52.806681    4473 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9dc3b95a-13ab-453d-a5a5-d9037ebcfada-workload-socket\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:16:52.806782 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:52.806698    4473 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9dc3b95a-13ab-453d-a5a5-d9037ebcfada-credential-socket\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:16:52.807886 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:52.807860    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9dc3b95a-13ab-453d-a5a5-d9037ebcfada-istio-envoy" pod "9dc3b95a-13ab-453d-a5a5-d9037ebcfada" (UID: "9dc3b95a-13ab-453d-a5a5-d9037ebcfada"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:16:52.808203 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:52.808179    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9dc3b95a-13ab-453d-a5a5-d9037ebcfada-kube-api-access-dxjxj" pod "9dc3b95a-13ab-453d-a5a5-d9037ebcfada" (UID: "9dc3b95a-13ab-453d-a5a5-d9037ebcfada"). InnerVolumeSpecName "kube-api-access-dxjxj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:16:52.808315 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:52.808274    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9dc3b95a-13ab-453d-a5a5-d9037ebcfada-istio-token" pod "9dc3b95a-13ab-453d-a5a5-d9037ebcfada" (UID: "9dc3b95a-13ab-453d-a5a5-d9037ebcfada"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:16:52.808652 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:52.808634    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9dc3b95a-13ab-453d-a5a5-d9037ebcfada-istio-podinfo" pod "9dc3b95a-13ab-453d-a5a5-d9037ebcfada" (UID: "9dc3b95a-13ab-453d-a5a5-d9037ebcfada"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:16:52.949487 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:52.912052    4473 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9dc3b95a-13ab-453d-a5a5-d9037ebcfada-istio-envoy\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:16:52.949487 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:52.912093    4473 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dxjxj\" (UniqueName: \"kubernetes.io/projected/9dc3b95a-13ab-453d-a5a5-d9037ebcfada-kube-api-access-dxjxj\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:16:52.949487 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:52.912110    4473 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9dc3b95a-13ab-453d-a5a5-d9037ebcfada-istio-token\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:16:52.949487 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:52.912123    4473 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9dc3b95a-13ab-453d-a5a5-d9037ebcfada-istio-podinfo\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:16:53.025002 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:53.023373    4473 generic.go:356] "Generic (PLEG): container finished" podID="9dc3b95a-13ab-453d-a5a5-d9037ebcfada" containerID="e932b7049a0f917709c2793d151e7e4e8cce450cf4c793d2379ae3c806f88391" exitCode=0
Oct 30 22:16:53.025002 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:53.023538    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-dd67d" event={"ID":"9dc3b95a-13ab-453d-a5a5-d9037ebcfada","Type":"ContainerDied","Data":"e932b7049a0f917709c2793d151e7e4e8cce450cf4c793d2379ae3c806f88391"}
Oct 30 22:16:53.025002 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:53.023616    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-dd67d" event={"ID":"9dc3b95a-13ab-453d-a5a5-d9037ebcfada","Type":"ContainerDied","Data":"110aa388ca17fdd00f4057aafb86e0613da00b464fdd603cbf3702b4d8c70d75"}
Oct 30 22:16:53.025002 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:53.023768    4473 scope.go:122] "RemoveContainer" containerID="e932b7049a0f917709c2793d151e7e4e8cce450cf4c793d2379ae3c806f88391"
Oct 30 22:16:53.025002 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:53.023889    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-dd67d"
Oct 30 22:16:53.062101 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:53.062076    4473 scope.go:122] "RemoveContainer" containerID="e932b7049a0f917709c2793d151e7e4e8cce450cf4c793d2379ae3c806f88391"
Oct 30 22:16:53.062669 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:16:53.062578    4473 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e932b7049a0f917709c2793d151e7e4e8cce450cf4c793d2379ae3c806f88391\": container with ID starting with e932b7049a0f917709c2793d151e7e4e8cce450cf4c793d2379ae3c806f88391 not found: ID does not exist" containerID="e932b7049a0f917709c2793d151e7e4e8cce450cf4c793d2379ae3c806f88391"
Oct 30 22:16:53.062669 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:53.062620    4473 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e932b7049a0f917709c2793d151e7e4e8cce450cf4c793d2379ae3c806f88391"} err="failed to get container status \"e932b7049a0f917709c2793d151e7e4e8cce450cf4c793d2379ae3c806f88391\": rpc error: code = NotFound desc = could not find container \"e932b7049a0f917709c2793d151e7e4e8cce450cf4c793d2379ae3c806f88391\": container with ID starting with e932b7049a0f917709c2793d151e7e4e8cce450cf4c793d2379ae3c806f88391 not found: ID does not exist"
Oct 30 22:16:53.205690 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:53.204635    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-dd67d"]
Oct 30 22:16:53.210668 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:53.210639    4473 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-dd67d"]
Oct 30 22:16:54.361186 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:54.361148    4473 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9dc3b95a-13ab-453d-a5a5-d9037ebcfada" path="/var/lib/kubelet/pods/9dc3b95a-13ab-453d-a5a5-d9037ebcfada/volumes"
Oct 30 22:16:54.651306 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:54.651266    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-d516c4-687f696c99-xjfqk"]
Oct 30 22:16:54.651587 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:54.651558    4473 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-d516c4-687f696c99-xjfqk" podUID="164e37d6-3d6f-4210-974d-6e6016c97b40" containerName="maas-api" containerID="cri-o://161e4dd435e6d28824acd86fa28f5ae11dd6313527b49c13db89e9abc1fe1ad3" gracePeriod=30
Oct 30 22:16:55.002748 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:55.002722    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-d516c4-687f696c99-xjfqk"
Oct 30 22:16:55.044510 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:55.044474    4473 generic.go:356] "Generic (PLEG): container finished" podID="164e37d6-3d6f-4210-974d-6e6016c97b40" containerID="161e4dd435e6d28824acd86fa28f5ae11dd6313527b49c13db89e9abc1fe1ad3" exitCode=0
Oct 30 22:16:55.044669 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:55.044523    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-d516c4-687f696c99-xjfqk"
Oct 30 22:16:55.044669 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:55.044568    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-d516c4-687f696c99-xjfqk" event={"ID":"164e37d6-3d6f-4210-974d-6e6016c97b40","Type":"ContainerDied","Data":"161e4dd435e6d28824acd86fa28f5ae11dd6313527b49c13db89e9abc1fe1ad3"}
Oct 30 22:16:55.044669 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:55.044631    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-d516c4-687f696c99-xjfqk" event={"ID":"164e37d6-3d6f-4210-974d-6e6016c97b40","Type":"ContainerDied","Data":"24f2d99b98c5c29398324201a7e405e8b5aef47b022079208106b0a1df77f017"}
Oct 30 22:16:55.044669 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:55.044656    4473 scope.go:122] "RemoveContainer" containerID="161e4dd435e6d28824acd86fa28f5ae11dd6313527b49c13db89e9abc1fe1ad3"
Oct 30 22:16:55.124948 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:55.122133    4473 scope.go:122] "RemoveContainer" containerID="161e4dd435e6d28824acd86fa28f5ae11dd6313527b49c13db89e9abc1fe1ad3"
Oct 30 22:16:55.124948 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:55.123393    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/164e37d6-3d6f-4210-974d-6e6016c97b40-metrics-certs\" (UniqueName: \"kubernetes.io/secret/164e37d6-3d6f-4210-974d-6e6016c97b40-metrics-certs\") pod \"164e37d6-3d6f-4210-974d-6e6016c97b40\" (UID: \"164e37d6-3d6f-4210-974d-6e6016c97b40\") "
Oct 30 22:16:55.124948 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:55.123713    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/164e37d6-3d6f-4210-974d-6e6016c97b40-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/164e37d6-3d6f-4210-974d-6e6016c97b40-openshift-service-ca\") pod \"164e37d6-3d6f-4210-974d-6e6016c97b40\" (UID: \"164e37d6-3d6f-4210-974d-6e6016c97b40\") "
Oct 30 22:16:55.148943 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:55.138547    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/164e37d6-3d6f-4210-974d-6e6016c97b40-kube-api-access-2l68v\" (UniqueName: \"kubernetes.io/projected/164e37d6-3d6f-4210-974d-6e6016c97b40-kube-api-access-2l68v\") pod \"164e37d6-3d6f-4210-974d-6e6016c97b40\" (UID: \"164e37d6-3d6f-4210-974d-6e6016c97b40\") "
Oct 30 22:16:55.148943 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:55.138627    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/164e37d6-3d6f-4210-974d-6e6016c97b40-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/164e37d6-3d6f-4210-974d-6e6016c97b40-maas-api-tls\") pod \"164e37d6-3d6f-4210-974d-6e6016c97b40\" (UID: \"164e37d6-3d6f-4210-974d-6e6016c97b40\") "
Oct 30 22:16:55.148943 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:55.145700    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/164e37d6-3d6f-4210-974d-6e6016c97b40-metrics-certs" pod "164e37d6-3d6f-4210-974d-6e6016c97b40" (UID: "164e37d6-3d6f-4210-974d-6e6016c97b40"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:16:55.148943 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:55.146051    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/164e37d6-3d6f-4210-974d-6e6016c97b40-openshift-service-ca" pod "164e37d6-3d6f-4210-974d-6e6016c97b40" (UID: "164e37d6-3d6f-4210-974d-6e6016c97b40"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:16:55.149559 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:16:55.149515    4473 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"161e4dd435e6d28824acd86fa28f5ae11dd6313527b49c13db89e9abc1fe1ad3\": container with ID starting with 161e4dd435e6d28824acd86fa28f5ae11dd6313527b49c13db89e9abc1fe1ad3 not found: ID does not exist" containerID="161e4dd435e6d28824acd86fa28f5ae11dd6313527b49c13db89e9abc1fe1ad3"
Oct 30 22:16:55.149682 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:55.149574    4473 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"161e4dd435e6d28824acd86fa28f5ae11dd6313527b49c13db89e9abc1fe1ad3"} err="failed to get container status \"161e4dd435e6d28824acd86fa28f5ae11dd6313527b49c13db89e9abc1fe1ad3\": rpc error: code = NotFound desc = could not find container \"161e4dd435e6d28824acd86fa28f5ae11dd6313527b49c13db89e9abc1fe1ad3\": container with ID starting with 161e4dd435e6d28824acd86fa28f5ae11dd6313527b49c13db89e9abc1fe1ad3 not found: ID does not exist"
Oct 30 22:16:55.152427 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:55.150249    4473 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/164e37d6-3d6f-4210-974d-6e6016c97b40-openshift-service-ca\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:16:55.152427 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:55.150283    4473 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/164e37d6-3d6f-4210-974d-6e6016c97b40-metrics-certs\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:16:55.152427 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:55.150386    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/164e37d6-3d6f-4210-974d-6e6016c97b40-maas-api-tls" pod "164e37d6-3d6f-4210-974d-6e6016c97b40" (UID: "164e37d6-3d6f-4210-974d-6e6016c97b40"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:16:55.152427 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:55.151588    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/164e37d6-3d6f-4210-974d-6e6016c97b40-kube-api-access-2l68v" pod "164e37d6-3d6f-4210-974d-6e6016c97b40" (UID: "164e37d6-3d6f-4210-974d-6e6016c97b40"). InnerVolumeSpecName "kube-api-access-2l68v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:16:55.264533 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:55.263396    4473 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2l68v\" (UniqueName: \"kubernetes.io/projected/164e37d6-3d6f-4210-974d-6e6016c97b40-kube-api-access-2l68v\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:16:55.264533 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:55.263450    4473 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/164e37d6-3d6f-4210-974d-6e6016c97b40-maas-api-tls\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:16:55.468782 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:55.468726    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-d516c4-687f696c99-xjfqk"]
Oct 30 22:16:55.471892 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:55.471863    4473 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-d516c4-687f696c99-xjfqk"]
Oct 30 22:16:55.965336 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:55.965297    4473 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w6-0a7679-69cc484995-hlw4x" podUID="9e922331-4f21-49d7-88d3-7a48d2854cea" containerName="payload-pre-processing" containerID="cri-o://73b26cc4e4691e234b566f9b4a1c515a22c1ef52aba646cfbeb66c788459b03c" gracePeriod=25
Oct 30 22:16:56.235839 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:56.235771    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w6-0a7679-69cc484995-hlw4x"
Oct 30 22:16:56.283860 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:56.283823    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9e922331-4f21-49d7-88d3-7a48d2854cea-kube-api-access-d7wxt\" (UniqueName: \"kubernetes.io/projected/9e922331-4f21-49d7-88d3-7a48d2854cea-kube-api-access-d7wxt\") pod \"9e922331-4f21-49d7-88d3-7a48d2854cea\" (UID: \"9e922331-4f21-49d7-88d3-7a48d2854cea\") "
Oct 30 22:16:56.284152 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:56.284119    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9e922331-4f21-49d7-88d3-7a48d2854cea-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9e922331-4f21-49d7-88d3-7a48d2854cea-plugins-config-volume\") pod \"9e922331-4f21-49d7-88d3-7a48d2854cea\" (UID: \"9e922331-4f21-49d7-88d3-7a48d2854cea\") "
Oct 30 22:16:56.284694 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:56.284661    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9e922331-4f21-49d7-88d3-7a48d2854cea-plugins-config-volume" pod "9e922331-4f21-49d7-88d3-7a48d2854cea" (UID: "9e922331-4f21-49d7-88d3-7a48d2854cea"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:16:56.286881 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:56.286856    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9e922331-4f21-49d7-88d3-7a48d2854cea-kube-api-access-d7wxt" pod "9e922331-4f21-49d7-88d3-7a48d2854cea" (UID: "9e922331-4f21-49d7-88d3-7a48d2854cea"). InnerVolumeSpecName "kube-api-access-d7wxt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:16:56.373387 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:56.373346    4473 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="164e37d6-3d6f-4210-974d-6e6016c97b40" path="/var/lib/kubelet/pods/164e37d6-3d6f-4210-974d-6e6016c97b40/volumes"
Oct 30 22:16:56.385877 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:56.385843    4473 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d7wxt\" (UniqueName: \"kubernetes.io/projected/9e922331-4f21-49d7-88d3-7a48d2854cea-kube-api-access-d7wxt\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:16:56.386044 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:56.385882    4473 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9e922331-4f21-49d7-88d3-7a48d2854cea-plugins-config-volume\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:16:57.060796 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:57.060765    4473 generic.go:356] "Generic (PLEG): container finished" podID="9e922331-4f21-49d7-88d3-7a48d2854cea" containerID="73b26cc4e4691e234b566f9b4a1c515a22c1ef52aba646cfbeb66c788459b03c" exitCode=0
Oct 30 22:16:57.061347 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:57.060845    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w6-0a7679-69cc484995-hlw4x"
Oct 30 22:16:57.061347 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:57.060859    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w6-0a7679-69cc484995-hlw4x" event={"ID":"9e922331-4f21-49d7-88d3-7a48d2854cea","Type":"ContainerDied","Data":"73b26cc4e4691e234b566f9b4a1c515a22c1ef52aba646cfbeb66c788459b03c"}
Oct 30 22:16:57.061347 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:57.060969    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w6-0a7679-69cc484995-hlw4x" event={"ID":"9e922331-4f21-49d7-88d3-7a48d2854cea","Type":"ContainerDied","Data":"a8cd11d0384d99aaa3d06cd1707ffbe46ad65f570a94c9601b80563662f9f328"}
Oct 30 22:16:57.061347 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:57.061005    4473 scope.go:122] "RemoveContainer" containerID="73b26cc4e4691e234b566f9b4a1c515a22c1ef52aba646cfbeb66c788459b03c"
Oct 30 22:16:57.071627 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:57.071604    4473 scope.go:122] "RemoveContainer" containerID="73b26cc4e4691e234b566f9b4a1c515a22c1ef52aba646cfbeb66c788459b03c"
Oct 30 22:16:57.071984 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:16:57.071962    4473 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"73b26cc4e4691e234b566f9b4a1c515a22c1ef52aba646cfbeb66c788459b03c\": container with ID starting with 73b26cc4e4691e234b566f9b4a1c515a22c1ef52aba646cfbeb66c788459b03c not found: ID does not exist" containerID="73b26cc4e4691e234b566f9b4a1c515a22c1ef52aba646cfbeb66c788459b03c"
Oct 30 22:16:57.072047 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:57.071996    4473 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"73b26cc4e4691e234b566f9b4a1c515a22c1ef52aba646cfbeb66c788459b03c"} err="failed to get container status \"73b26cc4e4691e234b566f9b4a1c515a22c1ef52aba646cfbeb66c788459b03c\": rpc error: code = NotFound desc = could not find container \"73b26cc4e4691e234b566f9b4a1c515a22c1ef52aba646cfbeb66c788459b03c\": container with ID starting with 73b26cc4e4691e234b566f9b4a1c515a22c1ef52aba646cfbeb66c788459b03c not found: ID does not exist"
Oct 30 22:16:57.233941 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:57.233894    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w6-0a7679-69cc484995-hlw4x"]
Oct 30 22:16:57.241913 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:57.241879    4473 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w6-0a7679-69cc484995-hlw4x"]
Oct 30 22:16:58.359407 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:16:58.359374    4473 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9e922331-4f21-49d7-88d3-7a48d2854cea" path="/var/lib/kubelet/pods/9e922331-4f21-49d7-88d3-7a48d2854cea/volumes"
Oct 30 22:17:00.959230 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:00.959196    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-242ddaf0-5d8df4c987-fkjbf"]
Oct 30 22:17:00.959640 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:00.959408    4473 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-242ddaf0-5d8df4c987-fkjbf" podUID="d763728d-389e-4e4d-8337-f9ae9d09478f" containerName="maas-api" containerID="cri-o://fc6c6667ac9d2df46711e4ea6ee3327d3d2c7eadcbe6155844ec132ad78c1f57" gracePeriod=30
Oct 30 22:17:01.091260 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:01.091226    4473 generic.go:356] "Generic (PLEG): container finished" podID="d763728d-389e-4e4d-8337-f9ae9d09478f" containerID="fc6c6667ac9d2df46711e4ea6ee3327d3d2c7eadcbe6155844ec132ad78c1f57" exitCode=0
Oct 30 22:17:01.091435 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:01.091301    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-242ddaf0-5d8df4c987-fkjbf" event={"ID":"d763728d-389e-4e4d-8337-f9ae9d09478f","Type":"ContainerDied","Data":"fc6c6667ac9d2df46711e4ea6ee3327d3d2c7eadcbe6155844ec132ad78c1f57"}
Oct 30 22:17:01.294756 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:01.294731    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-242ddaf0-5d8df4c987-fkjbf"
Oct 30 22:17:01.362590 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:01.362543    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d763728d-389e-4e4d-8337-f9ae9d09478f-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d763728d-389e-4e4d-8337-f9ae9d09478f-maas-api-tls\") pod \"d763728d-389e-4e4d-8337-f9ae9d09478f\" (UID: \"d763728d-389e-4e4d-8337-f9ae9d09478f\") "
Oct 30 22:17:01.371111 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:01.371073    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d763728d-389e-4e4d-8337-f9ae9d09478f-metrics-certs\" (UniqueName: \"kubernetes.io/secret/d763728d-389e-4e4d-8337-f9ae9d09478f-metrics-certs\") pod \"d763728d-389e-4e4d-8337-f9ae9d09478f\" (UID: \"d763728d-389e-4e4d-8337-f9ae9d09478f\") "
Oct 30 22:17:01.371363 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:01.371344    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d763728d-389e-4e4d-8337-f9ae9d09478f-kube-api-access-mp5zr\" (UniqueName: \"kubernetes.io/projected/d763728d-389e-4e4d-8337-f9ae9d09478f-kube-api-access-mp5zr\") pod \"d763728d-389e-4e4d-8337-f9ae9d09478f\" (UID: \"d763728d-389e-4e4d-8337-f9ae9d09478f\") "
Oct 30 22:17:01.371545 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:01.371529    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d763728d-389e-4e4d-8337-f9ae9d09478f-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d763728d-389e-4e4d-8337-f9ae9d09478f-openshift-service-ca\") pod \"d763728d-389e-4e4d-8337-f9ae9d09478f\" (UID: \"d763728d-389e-4e4d-8337-f9ae9d09478f\") "
Oct 30 22:17:01.398222 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:01.388727    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d763728d-389e-4e4d-8337-f9ae9d09478f-maas-api-tls" pod "d763728d-389e-4e4d-8337-f9ae9d09478f" (UID: "d763728d-389e-4e4d-8337-f9ae9d09478f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:17:01.398222 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:01.390759    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d763728d-389e-4e4d-8337-f9ae9d09478f-metrics-certs" pod "d763728d-389e-4e4d-8337-f9ae9d09478f" (UID: "d763728d-389e-4e4d-8337-f9ae9d09478f"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:17:01.414640 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:01.414599    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d763728d-389e-4e4d-8337-f9ae9d09478f-kube-api-access-mp5zr" pod "d763728d-389e-4e4d-8337-f9ae9d09478f" (UID: "d763728d-389e-4e4d-8337-f9ae9d09478f"). InnerVolumeSpecName "kube-api-access-mp5zr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:17:01.436413 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:01.436369    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d763728d-389e-4e4d-8337-f9ae9d09478f-openshift-service-ca" pod "d763728d-389e-4e4d-8337-f9ae9d09478f" (UID: "d763728d-389e-4e4d-8337-f9ae9d09478f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:17:01.450775 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:01.450733    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w2-0c99b1-64b58dc987-5h7bc" podUID="405c4ba4-c940-45e9-bcb9-484428bf5c5b" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:17:01.473812 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:01.473771    4473 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d763728d-389e-4e4d-8337-f9ae9d09478f-maas-api-tls\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:17:01.473812 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:01.473804    4473 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d763728d-389e-4e4d-8337-f9ae9d09478f-metrics-certs\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:17:01.473812 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:01.473818    4473 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mp5zr\" (UniqueName: \"kubernetes.io/projected/d763728d-389e-4e4d-8337-f9ae9d09478f-kube-api-access-mp5zr\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:17:01.474190 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:01.473833    4473 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d763728d-389e-4e4d-8337-f9ae9d09478f-openshift-service-ca\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:17:02.099261 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:02.099236    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-242ddaf0-5d8df4c987-fkjbf"
Oct 30 22:17:02.099762 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:02.099229    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-242ddaf0-5d8df4c987-fkjbf" event={"ID":"d763728d-389e-4e4d-8337-f9ae9d09478f","Type":"ContainerDied","Data":"86d498560dfd089c6e643ded88f943f1c28e2e490ca2cbb7c405b70b4cd461fd"}
Oct 30 22:17:02.099762 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:02.099376    4473 scope.go:122] "RemoveContainer" containerID="fc6c6667ac9d2df46711e4ea6ee3327d3d2c7eadcbe6155844ec132ad78c1f57"
Oct 30 22:17:02.270551 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:02.270463    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-242ddaf0-5d8df4c987-fkjbf"]
Oct 30 22:17:02.321029 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:02.320987    4473 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-242ddaf0-5d8df4c987-fkjbf"]
Oct 30 22:17:02.358434 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:02.358396    4473 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d763728d-389e-4e4d-8337-f9ae9d09478f" path="/var/lib/kubelet/pods/d763728d-389e-4e4d-8337-f9ae9d09478f/volumes"
Oct 30 22:17:02.817304 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:02.817235    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w2-0c99b1-64b58dc987-5h7bc"
Oct 30 22:17:02.897763 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:02.897731    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/405c4ba4-c940-45e9-bcb9-484428bf5c5b-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/405c4ba4-c940-45e9-bcb9-484428bf5c5b-plugins-config-volume\") pod \"405c4ba4-c940-45e9-bcb9-484428bf5c5b\" (UID: \"405c4ba4-c940-45e9-bcb9-484428bf5c5b\") "
Oct 30 22:17:02.898078 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:02.898057    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/405c4ba4-c940-45e9-bcb9-484428bf5c5b-kube-api-access-nxvzp\" (UniqueName: \"kubernetes.io/projected/405c4ba4-c940-45e9-bcb9-484428bf5c5b-kube-api-access-nxvzp\") pod \"405c4ba4-c940-45e9-bcb9-484428bf5c5b\" (UID: \"405c4ba4-c940-45e9-bcb9-484428bf5c5b\") "
Oct 30 22:17:02.898170 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:02.898138    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/405c4ba4-c940-45e9-bcb9-484428bf5c5b-plugins-config-volume" pod "405c4ba4-c940-45e9-bcb9-484428bf5c5b" (UID: "405c4ba4-c940-45e9-bcb9-484428bf5c5b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:17:02.898519 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:02.898499    4473 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/405c4ba4-c940-45e9-bcb9-484428bf5c5b-plugins-config-volume\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:17:02.900342 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:02.900321    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/405c4ba4-c940-45e9-bcb9-484428bf5c5b-kube-api-access-nxvzp" pod "405c4ba4-c940-45e9-bcb9-484428bf5c5b" (UID: "405c4ba4-c940-45e9-bcb9-484428bf5c5b"). InnerVolumeSpecName "kube-api-access-nxvzp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:17:03.000822 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:03.000782    4473 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nxvzp\" (UniqueName: \"kubernetes.io/projected/405c4ba4-c940-45e9-bcb9-484428bf5c5b-kube-api-access-nxvzp\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:17:03.119773 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:03.117393    4473 generic.go:356] "Generic (PLEG): container finished" podID="405c4ba4-c940-45e9-bcb9-484428bf5c5b" containerID="01b67c114f1f80a3d77401e5a41c2dbea7e7cc1e1f2059a69a7093920ba28150" exitCode=0
Oct 30 22:17:03.119773 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:03.117549    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w2-0c99b1-64b58dc987-5h7bc" event={"ID":"405c4ba4-c940-45e9-bcb9-484428bf5c5b","Type":"ContainerDied","Data":"01b67c114f1f80a3d77401e5a41c2dbea7e7cc1e1f2059a69a7093920ba28150"}
Oct 30 22:17:03.119773 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:03.117623    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w2-0c99b1-64b58dc987-5h7bc" event={"ID":"405c4ba4-c940-45e9-bcb9-484428bf5c5b","Type":"ContainerDied","Data":"28fcbe78525a9b81063cec44ffa7d24b25e38ea09594d2bbd07976374d0b1c10"}
Oct 30 22:17:03.119773 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:03.117658    4473 scope.go:122] "RemoveContainer" containerID="01b67c114f1f80a3d77401e5a41c2dbea7e7cc1e1f2059a69a7093920ba28150"
Oct 30 22:17:03.119773 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:03.117802    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w2-0c99b1-64b58dc987-5h7bc"
Oct 30 22:17:03.156546 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:03.156494    4473 scope.go:122] "RemoveContainer" containerID="01b67c114f1f80a3d77401e5a41c2dbea7e7cc1e1f2059a69a7093920ba28150"
Oct 30 22:17:03.157513 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:17:03.157479    4473 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"01b67c114f1f80a3d77401e5a41c2dbea7e7cc1e1f2059a69a7093920ba28150\": container with ID starting with 01b67c114f1f80a3d77401e5a41c2dbea7e7cc1e1f2059a69a7093920ba28150 not found: ID does not exist" containerID="01b67c114f1f80a3d77401e5a41c2dbea7e7cc1e1f2059a69a7093920ba28150"
Oct 30 22:17:03.157705 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:03.157668    4473 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"01b67c114f1f80a3d77401e5a41c2dbea7e7cc1e1f2059a69a7093920ba28150"} err="failed to get container status \"01b67c114f1f80a3d77401e5a41c2dbea7e7cc1e1f2059a69a7093920ba28150\": rpc error: code = NotFound desc = could not find container \"01b67c114f1f80a3d77401e5a41c2dbea7e7cc1e1f2059a69a7093920ba28150\": container with ID starting with 01b67c114f1f80a3d77401e5a41c2dbea7e7cc1e1f2059a69a7093920ba28150 not found: ID does not exist"
Oct 30 22:17:03.291303 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:03.291260    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w2-0c99b1-64b58dc987-5h7bc"]
Oct 30 22:17:03.297827 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:03.297797    4473 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w2-0c99b1-64b58dc987-5h7bc"]
Oct 30 22:17:03.833291 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:03.833245    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-tjjpc"]
Oct 30 22:17:03.833527 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:03.833495    4473 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-tjjpc" podUID="a3128eb9-83aa-4ea0-abaf-c642b921870a" containerName="istio-proxy" containerID="cri-o://0be752dc7462fe8fa74ef5ed5ad803a427ac7f621db2e23cb25d2cce2e34d9ad" gracePeriod=30
Oct 30 22:17:04.358161 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:04.358125    4473 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="405c4ba4-c940-45e9-bcb9-484428bf5c5b" path="/var/lib/kubelet/pods/405c4ba4-c940-45e9-bcb9-484428bf5c5b/volumes"
Oct 30 22:17:07.904375 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:07.904342    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-ebddcc-867b88b67c-b66lr"]
Oct 30 22:17:07.904971 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:07.904539    4473 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-ebddcc-867b88b67c-b66lr" podUID="4c3f076c-c17f-4c7a-af38-10a974ced61d" containerName="maas-api" containerID="cri-o://7397a43ed567843738c486f5e168ace0a2b51d64c1045e46486e1ba5430e2ed3" gracePeriod=30
Oct 30 22:17:08.182047 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:08.171356    4473 generic.go:356] "Generic (PLEG): container finished" podID="4c3f076c-c17f-4c7a-af38-10a974ced61d" containerID="7397a43ed567843738c486f5e168ace0a2b51d64c1045e46486e1ba5430e2ed3" exitCode=0
Oct 30 22:17:08.182047 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:08.171494    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-ebddcc-867b88b67c-b66lr" event={"ID":"4c3f076c-c17f-4c7a-af38-10a974ced61d","Type":"ContainerDied","Data":"7397a43ed567843738c486f5e168ace0a2b51d64c1045e46486e1ba5430e2ed3"}
Oct 30 22:17:08.248661 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:08.248635    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-ebddcc-867b88b67c-b66lr"
Oct 30 22:17:08.278785 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:08.278750    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4c3f076c-c17f-4c7a-af38-10a974ced61d-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4c3f076c-c17f-4c7a-af38-10a974ced61d-maas-api-tls\") pod \"4c3f076c-c17f-4c7a-af38-10a974ced61d\" (UID: \"4c3f076c-c17f-4c7a-af38-10a974ced61d\") "
Oct 30 22:17:08.279008 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:08.278799    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4c3f076c-c17f-4c7a-af38-10a974ced61d-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4c3f076c-c17f-4c7a-af38-10a974ced61d-openshift-service-ca\") pod \"4c3f076c-c17f-4c7a-af38-10a974ced61d\" (UID: \"4c3f076c-c17f-4c7a-af38-10a974ced61d\") "
Oct 30 22:17:08.279008 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:08.278832    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4c3f076c-c17f-4c7a-af38-10a974ced61d-kube-api-access-pvcwh\" (UniqueName: \"kubernetes.io/projected/4c3f076c-c17f-4c7a-af38-10a974ced61d-kube-api-access-pvcwh\") pod \"4c3f076c-c17f-4c7a-af38-10a974ced61d\" (UID: \"4c3f076c-c17f-4c7a-af38-10a974ced61d\") "
Oct 30 22:17:08.279008 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:08.278984    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4c3f076c-c17f-4c7a-af38-10a974ced61d-metrics-certs\" (UniqueName: \"kubernetes.io/secret/4c3f076c-c17f-4c7a-af38-10a974ced61d-metrics-certs\") pod \"4c3f076c-c17f-4c7a-af38-10a974ced61d\" (UID: \"4c3f076c-c17f-4c7a-af38-10a974ced61d\") "
Oct 30 22:17:08.279284 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:08.279260    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4c3f076c-c17f-4c7a-af38-10a974ced61d-openshift-service-ca" pod "4c3f076c-c17f-4c7a-af38-10a974ced61d" (UID: "4c3f076c-c17f-4c7a-af38-10a974ced61d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:17:08.279591 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:08.279568    4473 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4c3f076c-c17f-4c7a-af38-10a974ced61d-openshift-service-ca\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:17:08.281324 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:08.281298    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4c3f076c-c17f-4c7a-af38-10a974ced61d-metrics-certs" pod "4c3f076c-c17f-4c7a-af38-10a974ced61d" (UID: "4c3f076c-c17f-4c7a-af38-10a974ced61d"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:17:08.281413 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:08.281373    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4c3f076c-c17f-4c7a-af38-10a974ced61d-kube-api-access-pvcwh" pod "4c3f076c-c17f-4c7a-af38-10a974ced61d" (UID: "4c3f076c-c17f-4c7a-af38-10a974ced61d"). InnerVolumeSpecName "kube-api-access-pvcwh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:17:08.281463 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:08.281447    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4c3f076c-c17f-4c7a-af38-10a974ced61d-maas-api-tls" pod "4c3f076c-c17f-4c7a-af38-10a974ced61d" (UID: "4c3f076c-c17f-4c7a-af38-10a974ced61d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:17:08.381825 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:08.381785    4473 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4c3f076c-c17f-4c7a-af38-10a974ced61d-maas-api-tls\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:17:08.381825 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:08.381826    4473 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pvcwh\" (UniqueName: \"kubernetes.io/projected/4c3f076c-c17f-4c7a-af38-10a974ced61d-kube-api-access-pvcwh\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:17:08.382065 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:08.381841    4473 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4c3f076c-c17f-4c7a-af38-10a974ced61d-metrics-certs\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:17:09.232046 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:09.231991    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-b0c55f-8457bc5476-5gwgt"]
Oct 30 22:17:09.232745 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:09.232237    4473 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-b0c55f-8457bc5476-5gwgt" podUID="895a52c4-b3cd-4cb0-8e80-46f08414b29f" containerName="maas-api" containerID="cri-o://aba9e873750983f6175d101713712392662f1320b48b139c2394c717a9203b84" gracePeriod=30
Oct 30 22:17:09.265836 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:09.264763    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-ebddcc-867b88b67c-b66lr" event={"ID":"4c3f076c-c17f-4c7a-af38-10a974ced61d","Type":"ContainerDied","Data":"dcb40eec6ba95e11954f88477fbcbdcbd936e849eb7e3a0b8912da4564d4b3e0"}
Oct 30 22:17:09.265836 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:09.264840    4473 scope.go:122] "RemoveContainer" containerID="7397a43ed567843738c486f5e168ace0a2b51d64c1045e46486e1ba5430e2ed3"
Oct 30 22:17:09.265836 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:09.265032    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-ebddcc-867b88b67c-b66lr"
Oct 30 22:17:09.277205 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:09.277166    4473 generic.go:356] "Generic (PLEG): container finished" podID="a3128eb9-83aa-4ea0-abaf-c642b921870a" containerID="0be752dc7462fe8fa74ef5ed5ad803a427ac7f621db2e23cb25d2cce2e34d9ad" exitCode=0
Oct 30 22:17:09.277357 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:09.277274    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-tjjpc" event={"ID":"a3128eb9-83aa-4ea0-abaf-c642b921870a","Type":"ContainerDied","Data":"0be752dc7462fe8fa74ef5ed5ad803a427ac7f621db2e23cb25d2cce2e34d9ad"}
Oct 30 22:17:09.404674 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:09.404649    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-tjjpc"
Oct 30 22:17:09.504587 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:09.499388    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-ebddcc-867b88b67c-b66lr"]
Oct 30 22:17:09.504587 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:09.504425    4473 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-ebddcc-867b88b67c-b66lr"]
Oct 30 22:17:09.542100 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:09.542064    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a3128eb9-83aa-4ea0-abaf-c642b921870a-kube-api-access-vj7qj\" (UniqueName: \"kubernetes.io/projected/a3128eb9-83aa-4ea0-abaf-c642b921870a-kube-api-access-vj7qj\") pod \"a3128eb9-83aa-4ea0-abaf-c642b921870a\" (UID: \"a3128eb9-83aa-4ea0-abaf-c642b921870a\") "
Oct 30 22:17:09.542306 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:09.542149    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a3128eb9-83aa-4ea0-abaf-c642b921870a-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a3128eb9-83aa-4ea0-abaf-c642b921870a-workload-certs\") pod \"a3128eb9-83aa-4ea0-abaf-c642b921870a\" (UID: \"a3128eb9-83aa-4ea0-abaf-c642b921870a\") "
Oct 30 22:17:09.542306 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:09.542180    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a3128eb9-83aa-4ea0-abaf-c642b921870a-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3128eb9-83aa-4ea0-abaf-c642b921870a-credential-socket\") pod \"a3128eb9-83aa-4ea0-abaf-c642b921870a\" (UID: \"a3128eb9-83aa-4ea0-abaf-c642b921870a\") "
Oct 30 22:17:09.542632 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:09.542601    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3128eb9-83aa-4ea0-abaf-c642b921870a-credential-socket" pod "a3128eb9-83aa-4ea0-abaf-c642b921870a" (UID: "a3128eb9-83aa-4ea0-abaf-c642b921870a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:17:09.542755 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:09.542687    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a3128eb9-83aa-4ea0-abaf-c642b921870a-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a3128eb9-83aa-4ea0-abaf-c642b921870a-istiod-ca-cert\") pod \"a3128eb9-83aa-4ea0-abaf-c642b921870a\" (UID: \"a3128eb9-83aa-4ea0-abaf-c642b921870a\") "
Oct 30 22:17:09.542819 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:09.542781    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3128eb9-83aa-4ea0-abaf-c642b921870a-workload-certs" pod "a3128eb9-83aa-4ea0-abaf-c642b921870a" (UID: "a3128eb9-83aa-4ea0-abaf-c642b921870a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:17:09.542876 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:09.542838    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/a3128eb9-83aa-4ea0-abaf-c642b921870a-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a3128eb9-83aa-4ea0-abaf-c642b921870a-istio-podinfo\") pod \"a3128eb9-83aa-4ea0-abaf-c642b921870a\" (UID: \"a3128eb9-83aa-4ea0-abaf-c642b921870a\") "
Oct 30 22:17:09.542978 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:09.542878    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a3128eb9-83aa-4ea0-abaf-c642b921870a-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a3128eb9-83aa-4ea0-abaf-c642b921870a-istio-envoy\") pod \"a3128eb9-83aa-4ea0-abaf-c642b921870a\" (UID: \"a3128eb9-83aa-4ea0-abaf-c642b921870a\") "
Oct 30 22:17:09.543045 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:09.543022    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a3128eb9-83aa-4ea0-abaf-c642b921870a-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3128eb9-83aa-4ea0-abaf-c642b921870a-workload-socket\") pod \"a3128eb9-83aa-4ea0-abaf-c642b921870a\" (UID: \"a3128eb9-83aa-4ea0-abaf-c642b921870a\") "
Oct 30 22:17:09.543112 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:09.543095    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a3128eb9-83aa-4ea0-abaf-c642b921870a-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a3128eb9-83aa-4ea0-abaf-c642b921870a-istio-data\") pod \"a3128eb9-83aa-4ea0-abaf-c642b921870a\" (UID: \"a3128eb9-83aa-4ea0-abaf-c642b921870a\") "
Oct 30 22:17:09.543173 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:09.543150    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a3128eb9-83aa-4ea0-abaf-c642b921870a-istio-token\" (UniqueName: \"kubernetes.io/projected/a3128eb9-83aa-4ea0-abaf-c642b921870a-istio-token\") pod \"a3128eb9-83aa-4ea0-abaf-c642b921870a\" (UID: \"a3128eb9-83aa-4ea0-abaf-c642b921870a\") "
Oct 30 22:17:09.543231 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:09.543167    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a3128eb9-83aa-4ea0-abaf-c642b921870a-istiod-ca-cert" pod "a3128eb9-83aa-4ea0-abaf-c642b921870a" (UID: "a3128eb9-83aa-4ea0-abaf-c642b921870a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:17:09.543608 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:09.543586    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3128eb9-83aa-4ea0-abaf-c642b921870a-istio-data" pod "a3128eb9-83aa-4ea0-abaf-c642b921870a" (UID: "a3128eb9-83aa-4ea0-abaf-c642b921870a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:17:09.543761 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:09.543746    4473 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a3128eb9-83aa-4ea0-abaf-c642b921870a-istio-data\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:17:09.543823 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:09.543767    4473 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a3128eb9-83aa-4ea0-abaf-c642b921870a-workload-certs\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:17:09.543823 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:09.543782    4473 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3128eb9-83aa-4ea0-abaf-c642b921870a-credential-socket\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:17:09.543823 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:09.543799    4473 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a3128eb9-83aa-4ea0-abaf-c642b921870a-istiod-ca-cert\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:17:09.544092 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:09.544070    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3128eb9-83aa-4ea0-abaf-c642b921870a-workload-socket" pod "a3128eb9-83aa-4ea0-abaf-c642b921870a" (UID: "a3128eb9-83aa-4ea0-abaf-c642b921870a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:17:09.546270 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:09.546241    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3128eb9-83aa-4ea0-abaf-c642b921870a-istio-envoy" pod "a3128eb9-83aa-4ea0-abaf-c642b921870a" (UID: "a3128eb9-83aa-4ea0-abaf-c642b921870a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:17:09.546524 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:09.546498    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a3128eb9-83aa-4ea0-abaf-c642b921870a-kube-api-access-vj7qj" pod "a3128eb9-83aa-4ea0-abaf-c642b921870a" (UID: "a3128eb9-83aa-4ea0-abaf-c642b921870a"). InnerVolumeSpecName "kube-api-access-vj7qj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:17:09.547643 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:09.547617    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a3128eb9-83aa-4ea0-abaf-c642b921870a-istio-token" pod "a3128eb9-83aa-4ea0-abaf-c642b921870a" (UID: "a3128eb9-83aa-4ea0-abaf-c642b921870a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:17:09.547741 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:09.547685    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a3128eb9-83aa-4ea0-abaf-c642b921870a-istio-podinfo" pod "a3128eb9-83aa-4ea0-abaf-c642b921870a" (UID: "a3128eb9-83aa-4ea0-abaf-c642b921870a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:17:09.580137 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:09.580109    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-b0c55f-8457bc5476-5gwgt"
Oct 30 22:17:09.658176 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:09.658050    4473 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3128eb9-83aa-4ea0-abaf-c642b921870a-workload-socket\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:17:09.658176 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:09.658092    4473 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a3128eb9-83aa-4ea0-abaf-c642b921870a-istio-token\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:17:09.658176 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:09.658108    4473 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vj7qj\" (UniqueName: \"kubernetes.io/projected/a3128eb9-83aa-4ea0-abaf-c642b921870a-kube-api-access-vj7qj\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:17:09.658176 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:09.658123    4473 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a3128eb9-83aa-4ea0-abaf-c642b921870a-istio-podinfo\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:17:09.658176 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:09.658136    4473 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a3128eb9-83aa-4ea0-abaf-c642b921870a-istio-envoy\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:17:09.759090 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:09.758992    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/895a52c4-b3cd-4cb0-8e80-46f08414b29f-metrics-certs\" (UniqueName: \"kubernetes.io/secret/895a52c4-b3cd-4cb0-8e80-46f08414b29f-metrics-certs\") pod \"895a52c4-b3cd-4cb0-8e80-46f08414b29f\" (UID: \"895a52c4-b3cd-4cb0-8e80-46f08414b29f\") "
Oct 30 22:17:09.759279 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:09.759157    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/895a52c4-b3cd-4cb0-8e80-46f08414b29f-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/895a52c4-b3cd-4cb0-8e80-46f08414b29f-maas-api-tls\") pod \"895a52c4-b3cd-4cb0-8e80-46f08414b29f\" (UID: \"895a52c4-b3cd-4cb0-8e80-46f08414b29f\") "
Oct 30 22:17:09.759349 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:09.759325    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/895a52c4-b3cd-4cb0-8e80-46f08414b29f-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/895a52c4-b3cd-4cb0-8e80-46f08414b29f-openshift-service-ca\") pod \"895a52c4-b3cd-4cb0-8e80-46f08414b29f\" (UID: \"895a52c4-b3cd-4cb0-8e80-46f08414b29f\") "
Oct 30 22:17:09.759416 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:09.759385    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/895a52c4-b3cd-4cb0-8e80-46f08414b29f-kube-api-access-5jb6b\" (UniqueName: \"kubernetes.io/projected/895a52c4-b3cd-4cb0-8e80-46f08414b29f-kube-api-access-5jb6b\") pod \"895a52c4-b3cd-4cb0-8e80-46f08414b29f\" (UID: \"895a52c4-b3cd-4cb0-8e80-46f08414b29f\") "
Oct 30 22:17:09.760145 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:09.760108    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/895a52c4-b3cd-4cb0-8e80-46f08414b29f-openshift-service-ca" pod "895a52c4-b3cd-4cb0-8e80-46f08414b29f" (UID: "895a52c4-b3cd-4cb0-8e80-46f08414b29f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:17:09.761638 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:09.761604    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/895a52c4-b3cd-4cb0-8e80-46f08414b29f-metrics-certs" pod "895a52c4-b3cd-4cb0-8e80-46f08414b29f" (UID: "895a52c4-b3cd-4cb0-8e80-46f08414b29f"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:17:09.762184 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:09.762165    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/895a52c4-b3cd-4cb0-8e80-46f08414b29f-maas-api-tls" pod "895a52c4-b3cd-4cb0-8e80-46f08414b29f" (UID: "895a52c4-b3cd-4cb0-8e80-46f08414b29f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:17:09.762250 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:09.762225    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/895a52c4-b3cd-4cb0-8e80-46f08414b29f-kube-api-access-5jb6b" pod "895a52c4-b3cd-4cb0-8e80-46f08414b29f" (UID: "895a52c4-b3cd-4cb0-8e80-46f08414b29f"). InnerVolumeSpecName "kube-api-access-5jb6b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:17:09.864946 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:09.861989    4473 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/895a52c4-b3cd-4cb0-8e80-46f08414b29f-metrics-certs\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:17:09.864946 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:09.862027    4473 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/895a52c4-b3cd-4cb0-8e80-46f08414b29f-maas-api-tls\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:17:09.864946 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:09.862043    4473 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/895a52c4-b3cd-4cb0-8e80-46f08414b29f-openshift-service-ca\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:17:09.864946 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:09.862058    4473 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5jb6b\" (UniqueName: \"kubernetes.io/projected/895a52c4-b3cd-4cb0-8e80-46f08414b29f-kube-api-access-5jb6b\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:17:09.988560 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:09.988523    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w5-e431fc-74d48cd559-c7bht"]
Oct 30 22:17:10.286746 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:10.286716    4473 generic.go:356] "Generic (PLEG): container finished" podID="895a52c4-b3cd-4cb0-8e80-46f08414b29f" containerID="aba9e873750983f6175d101713712392662f1320b48b139c2394c717a9203b84" exitCode=0
Oct 30 22:17:10.287273 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:10.286770    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-b0c55f-8457bc5476-5gwgt"
Oct 30 22:17:10.287273 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:10.286798    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-b0c55f-8457bc5476-5gwgt" event={"ID":"895a52c4-b3cd-4cb0-8e80-46f08414b29f","Type":"ContainerDied","Data":"aba9e873750983f6175d101713712392662f1320b48b139c2394c717a9203b84"}
Oct 30 22:17:10.287273 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:10.286853    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-b0c55f-8457bc5476-5gwgt" event={"ID":"895a52c4-b3cd-4cb0-8e80-46f08414b29f","Type":"ContainerDied","Data":"5b426bfddcf39a292c9c2a1a2f0c3a5c54fba721718b74925316eaa3e3f01474"}
Oct 30 22:17:10.287273 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:10.286884    4473 scope.go:122] "RemoveContainer" containerID="aba9e873750983f6175d101713712392662f1320b48b139c2394c717a9203b84"
Oct 30 22:17:10.314142 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:10.312147    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-tjjpc"
Oct 30 22:17:10.314142 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:10.313066    4473 scope.go:122] "RemoveContainer" containerID="aba9e873750983f6175d101713712392662f1320b48b139c2394c717a9203b84"
Oct 30 22:17:10.314142 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:10.313623    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-tjjpc" event={"ID":"a3128eb9-83aa-4ea0-abaf-c642b921870a","Type":"ContainerDied","Data":"f29c55d9f25c533a01d631750d7fb9b88ac38bc8147ea751f1808a69b4992b84"}
Oct 30 22:17:10.430067 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:17:10.397656    4473 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aba9e873750983f6175d101713712392662f1320b48b139c2394c717a9203b84\": container with ID starting with aba9e873750983f6175d101713712392662f1320b48b139c2394c717a9203b84 not found: ID does not exist" containerID="aba9e873750983f6175d101713712392662f1320b48b139c2394c717a9203b84"
Oct 30 22:17:10.430067 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:10.397710    4473 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aba9e873750983f6175d101713712392662f1320b48b139c2394c717a9203b84"} err="failed to get container status \"aba9e873750983f6175d101713712392662f1320b48b139c2394c717a9203b84\": rpc error: code = NotFound desc = could not find container \"aba9e873750983f6175d101713712392662f1320b48b139c2394c717a9203b84\": container with ID starting with aba9e873750983f6175d101713712392662f1320b48b139c2394c717a9203b84 not found: ID does not exist"
Oct 30 22:17:10.430067 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:10.397737    4473 scope.go:122] "RemoveContainer" containerID="0be752dc7462fe8fa74ef5ed5ad803a427ac7f621db2e23cb25d2cce2e34d9ad"
Oct 30 22:17:10.526181 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:10.526147    4473 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4c3f076c-c17f-4c7a-af38-10a974ced61d" path="/var/lib/kubelet/pods/4c3f076c-c17f-4c7a-af38-10a974ced61d/volumes"
Oct 30 22:17:10.573685 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:10.573546    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-b0c55f-8457bc5476-5gwgt"]
Oct 30 22:17:10.578766 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:10.578727    4473 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-b0c55f-8457bc5476-5gwgt"]
Oct 30 22:17:10.744530 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:10.744486    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-tjjpc"]
Oct 30 22:17:10.751207 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:10.751174    4473 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-d516c4-openshift-default-6bd584745-tjjpc"]
Oct 30 22:17:12.357344 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:12.357311    4473 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="895a52c4-b3cd-4cb0-8e80-46f08414b29f" path="/var/lib/kubelet/pods/895a52c4-b3cd-4cb0-8e80-46f08414b29f/volumes"
Oct 30 22:17:12.357878 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:12.357850    4473 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a3128eb9-83aa-4ea0-abaf-c642b921870a" path="/var/lib/kubelet/pods/a3128eb9-83aa-4ea0-abaf-c642b921870a/volumes"
Oct 30 22:17:14.928733 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:14.928697    4473 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-242ddaf0-7b7b974bc-5rsm9"]
Oct 30 22:17:14.929487 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:14.929469    4473 cpu_manager.go:414] "containerMap: removing container" podUID="164e37d6-3d6f-4210-974d-6e6016c97b40" containerName="maas-api"
Oct 30 22:17:14.929535 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:14.929489    4473 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="164e37d6-3d6f-4210-974d-6e6016c97b40" containerName="maas-api"
Oct 30 22:17:14.929535 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:14.929501    4473 cpu_manager.go:414] "containerMap: removing container" podUID="f260821f-0996-4717-9f4f-c30f41c238cb" containerName="payload-pre-processing"
Oct 30 22:17:14.929535 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:14.929506    4473 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f260821f-0996-4717-9f4f-c30f41c238cb" containerName="payload-pre-processing"
Oct 30 22:17:14.929535 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:14.929522    4473 cpu_manager.go:414] "containerMap: removing container" podUID="d763728d-389e-4e4d-8337-f9ae9d09478f" containerName="maas-api"
Oct 30 22:17:14.929535 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:14.929527    4473 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d763728d-389e-4e4d-8337-f9ae9d09478f" containerName="maas-api"
Oct 30 22:17:14.929535 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:14.929534    4473 cpu_manager.go:414] "containerMap: removing container" podUID="9dc3b95a-13ab-453d-a5a5-d9037ebcfada" containerName="istio-proxy"
Oct 30 22:17:14.929535 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:14.929538    4473 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9dc3b95a-13ab-453d-a5a5-d9037ebcfada" containerName="istio-proxy"
Oct 30 22:17:14.929759 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:14.929545    4473 cpu_manager.go:414] "containerMap: removing container" podUID="4c3f076c-c17f-4c7a-af38-10a974ced61d" containerName="maas-api"
Oct 30 22:17:14.929759 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:14.929550    4473 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4c3f076c-c17f-4c7a-af38-10a974ced61d" containerName="maas-api"
Oct 30 22:17:14.929759 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:14.929559    4473 cpu_manager.go:414] "containerMap: removing container" podUID="895a52c4-b3cd-4cb0-8e80-46f08414b29f" containerName="maas-api"
Oct 30 22:17:14.929759 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:14.929563    4473 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="895a52c4-b3cd-4cb0-8e80-46f08414b29f" containerName="maas-api"
Oct 30 22:17:14.929759 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:14.929572    4473 cpu_manager.go:414] "containerMap: removing container" podUID="7ab05096-862b-41f1-92ba-1eebceb2c026" containerName="istio-proxy"
Oct 30 22:17:14.929759 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:14.929576    4473 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7ab05096-862b-41f1-92ba-1eebceb2c026" containerName="istio-proxy"
Oct 30 22:17:14.929759 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:14.929584    4473 cpu_manager.go:414] "containerMap: removing container" podUID="405c4ba4-c940-45e9-bcb9-484428bf5c5b" containerName="payload-processing"
Oct 30 22:17:14.929759 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:14.929589    4473 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="405c4ba4-c940-45e9-bcb9-484428bf5c5b" containerName="payload-processing"
Oct 30 22:17:14.929759 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:14.929601    4473 cpu_manager.go:414] "containerMap: removing container" podUID="feeeb03e-f851-4d1e-bcff-0a934226a2d1" containerName="istio-proxy"
Oct 30 22:17:14.929759 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:14.929605    4473 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="feeeb03e-f851-4d1e-bcff-0a934226a2d1" containerName="istio-proxy"
Oct 30 22:17:14.929759 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:14.929611    4473 cpu_manager.go:414] "containerMap: removing container" podUID="feeeb03e-f851-4d1e-bcff-0a934226a2d1" containerName="istio-proxy"
Oct 30 22:17:14.929759 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:14.929615    4473 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="feeeb03e-f851-4d1e-bcff-0a934226a2d1" containerName="istio-proxy"
Oct 30 22:17:14.929759 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:14.929626    4473 cpu_manager.go:414] "containerMap: removing container" podUID="a3128eb9-83aa-4ea0-abaf-c642b921870a" containerName="istio-proxy"
Oct 30 22:17:14.929759 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:14.929630    4473 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a3128eb9-83aa-4ea0-abaf-c642b921870a" containerName="istio-proxy"
Oct 30 22:17:14.929759 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:14.929637    4473 cpu_manager.go:414] "containerMap: removing container" podUID="e94e5f73-3534-41c8-aa76-38ff540bf455" containerName="payload-processing"
Oct 30 22:17:14.929759 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:14.929641    4473 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e94e5f73-3534-41c8-aa76-38ff540bf455" containerName="payload-processing"
Oct 30 22:17:14.929759 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:14.929649    4473 cpu_manager.go:414] "containerMap: removing container" podUID="2102f3d2-e20b-402d-81ec-d6b4923df356" containerName="payload-pre-processing"
Oct 30 22:17:14.929759 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:14.929654    4473 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2102f3d2-e20b-402d-81ec-d6b4923df356" containerName="payload-pre-processing"
Oct 30 22:17:14.929759 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:14.929661    4473 cpu_manager.go:414] "containerMap: removing container" podUID="fe2360b8-5fff-43b6-bfa7-9f3e86cdbbcf" containerName="maas-api"
Oct 30 22:17:14.929759 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:14.929665    4473 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fe2360b8-5fff-43b6-bfa7-9f3e86cdbbcf" containerName="maas-api"
Oct 30 22:17:14.929759 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:14.929675    4473 cpu_manager.go:414] "containerMap: removing container" podUID="89c09555-cf53-49a7-9c3b-95d2da2a293a" containerName="main"
Oct 30 22:17:14.929759 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:14.929680    4473 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="89c09555-cf53-49a7-9c3b-95d2da2a293a" containerName="main"
Oct 30 22:17:14.929759 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:14.929690    4473 cpu_manager.go:414] "containerMap: removing container" podUID="9e922331-4f21-49d7-88d3-7a48d2854cea" containerName="payload-pre-processing"
Oct 30 22:17:14.929759 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:14.929694    4473 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9e922331-4f21-49d7-88d3-7a48d2854cea" containerName="payload-pre-processing"
Oct 30 22:17:14.930538 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:14.929771    4473 memory_manager.go:365] "RemoveStaleState removing state" podUID="164e37d6-3d6f-4210-974d-6e6016c97b40" containerName="maas-api"
Oct 30 22:17:14.930538 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:14.929781    4473 memory_manager.go:365] "RemoveStaleState removing state" podUID="f260821f-0996-4717-9f4f-c30f41c238cb" containerName="payload-pre-processing"
Oct 30 22:17:14.930538 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:14.929790    4473 memory_manager.go:365] "RemoveStaleState removing state" podUID="d763728d-389e-4e4d-8337-f9ae9d09478f" containerName="maas-api"
Oct 30 22:17:14.930538 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:14.929795    4473 memory_manager.go:365] "RemoveStaleState removing state" podUID="9dc3b95a-13ab-453d-a5a5-d9037ebcfada" containerName="istio-proxy"
Oct 30 22:17:14.930538 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:14.929801    4473 memory_manager.go:365] "RemoveStaleState removing state" podUID="4c3f076c-c17f-4c7a-af38-10a974ced61d" containerName="maas-api"
Oct 30 22:17:14.930538 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:14.929807    4473 memory_manager.go:365] "RemoveStaleState removing state" podUID="895a52c4-b3cd-4cb0-8e80-46f08414b29f" containerName="maas-api"
Oct 30 22:17:14.930538 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:14.929812    4473 memory_manager.go:365] "RemoveStaleState removing state" podUID="7ab05096-862b-41f1-92ba-1eebceb2c026" containerName="istio-proxy"
Oct 30 22:17:14.930538 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:14.929819    4473 memory_manager.go:365] "RemoveStaleState removing state" podUID="405c4ba4-c940-45e9-bcb9-484428bf5c5b" containerName="payload-processing"
Oct 30 22:17:14.930538 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:14.929825    4473 memory_manager.go:365] "RemoveStaleState removing state" podUID="feeeb03e-f851-4d1e-bcff-0a934226a2d1" containerName="istio-proxy"
Oct 30 22:17:14.930538 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:14.929830    4473 memory_manager.go:365] "RemoveStaleState removing state" podUID="feeeb03e-f851-4d1e-bcff-0a934226a2d1" containerName="istio-proxy"
Oct 30 22:17:14.930538 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:14.929836    4473 memory_manager.go:365] "RemoveStaleState removing state" podUID="a3128eb9-83aa-4ea0-abaf-c642b921870a" containerName="istio-proxy"
Oct 30 22:17:14.930538 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:14.929842    4473 memory_manager.go:365] "RemoveStaleState removing state" podUID="e94e5f73-3534-41c8-aa76-38ff540bf455" containerName="payload-processing"
Oct 30 22:17:14.930538 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:14.929847    4473 memory_manager.go:365] "RemoveStaleState removing state" podUID="2102f3d2-e20b-402d-81ec-d6b4923df356" containerName="payload-pre-processing"
Oct 30 22:17:14.930538 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:14.929851    4473 memory_manager.go:365] "RemoveStaleState removing state" podUID="fe2360b8-5fff-43b6-bfa7-9f3e86cdbbcf" containerName="maas-api"
Oct 30 22:17:14.930538 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:14.929857    4473 memory_manager.go:365] "RemoveStaleState removing state" podUID="89c09555-cf53-49a7-9c3b-95d2da2a293a" containerName="main"
Oct 30 22:17:14.930538 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:14.929862    4473 memory_manager.go:365] "RemoveStaleState removing state" podUID="9e922331-4f21-49d7-88d3-7a48d2854cea" containerName="payload-pre-processing"
Oct 30 22:17:14.935142 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:14.935118    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-242ddaf0-7b7b974bc-5rsm9"
Oct 30 22:17:14.989626 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:14.989563    4473 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w5-e431fc-74d48cd559-c7bht" podUID="b78bb642-f809-4102-b4e5-ac485f1ca50d" containerName="payload-pre-processing" containerID="cri-o://ee015d5000a27c4f2af5043d5c85626434a172cf344e7389e85bfeb54dd244c0" gracePeriod=25
Oct 30 22:17:14.991066 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:14.991040    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-242ddaf0-7b7b974bc-5rsm9"]
Oct 30 22:17:15.018009 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:15.017979    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/706bd83a-953d-4b2e-8f1c-e51bf4343a19-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-242ddaf0-7b7b974bc-5rsm9\" (UID: \"706bd83a-953d-4b2e-8f1c-e51bf4343a19\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-242ddaf0-7b7b974bc-5rsm9"
Oct 30 22:17:15.018190 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:15.018167    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-czbtn\" (UniqueName: \"kubernetes.io/projected/706bd83a-953d-4b2e-8f1c-e51bf4343a19-kube-api-access-czbtn\") pod \"payload-pre-processing-e2e-mt-242ddaf0-7b7b974bc-5rsm9\" (UID: \"706bd83a-953d-4b2e-8f1c-e51bf4343a19\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-242ddaf0-7b7b974bc-5rsm9"
Oct 30 22:17:15.037588 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:15.037555    4473 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-242ddaf0\""
Oct 30 22:17:15.166825 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:15.166751    4473 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-242ddaf0-576ccc4d89-28sq2"]
Oct 30 22:17:15.170596 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:15.170555    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-czbtn\" (UniqueName: \"kubernetes.io/projected/706bd83a-953d-4b2e-8f1c-e51bf4343a19-kube-api-access-czbtn\") pod \"payload-pre-processing-e2e-mt-242ddaf0-7b7b974bc-5rsm9\" (UID: \"706bd83a-953d-4b2e-8f1c-e51bf4343a19\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-242ddaf0-7b7b974bc-5rsm9"
Oct 30 22:17:15.170797 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:15.170732    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/706bd83a-953d-4b2e-8f1c-e51bf4343a19-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-242ddaf0-7b7b974bc-5rsm9\" (UID: \"706bd83a-953d-4b2e-8f1c-e51bf4343a19\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-242ddaf0-7b7b974bc-5rsm9"
Oct 30 22:17:15.178383 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:15.178346    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/706bd83a-953d-4b2e-8f1c-e51bf4343a19-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-242ddaf0-7b7b974bc-5rsm9\" (UID: \"706bd83a-953d-4b2e-8f1c-e51bf4343a19\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-242ddaf0-7b7b974bc-5rsm9"
Oct 30 22:17:15.237485 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:15.237440    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-242ddaf0-576ccc4d89-28sq2"]
Oct 30 22:17:15.237653 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:15.237605    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-242ddaf0-576ccc4d89-28sq2"
Oct 30 22:17:15.250955 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:15.250572    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-czbtn\" (UniqueName: \"kubernetes.io/projected/706bd83a-953d-4b2e-8f1c-e51bf4343a19-kube-api-access-czbtn\") pod \"payload-pre-processing-e2e-mt-242ddaf0-7b7b974bc-5rsm9\" (UID: \"706bd83a-953d-4b2e-8f1c-e51bf4343a19\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-242ddaf0-7b7b974bc-5rsm9"
Oct 30 22:17:15.250955 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:15.250829    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-242ddaf0-7b7b974bc-5rsm9"
Oct 30 22:17:15.274736 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:15.274426    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cd728188-2246-4f1d-8f18-ec351a96b935-plugins-config-volume\") pod \"payload-processing-e2e-mt-242ddaf0-576ccc4d89-28sq2\" (UID: \"cd728188-2246-4f1d-8f18-ec351a96b935\") " pod="openshift-ingress/payload-processing-e2e-mt-242ddaf0-576ccc4d89-28sq2"
Oct 30 22:17:15.274736 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:15.274474    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v7vls\" (UniqueName: \"kubernetes.io/projected/cd728188-2246-4f1d-8f18-ec351a96b935-kube-api-access-v7vls\") pod \"payload-processing-e2e-mt-242ddaf0-576ccc4d89-28sq2\" (UID: \"cd728188-2246-4f1d-8f18-ec351a96b935\") " pod="openshift-ingress/payload-processing-e2e-mt-242ddaf0-576ccc4d89-28sq2"
Oct 30 22:17:15.291087 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:15.290979    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-242ddaf0-dockercfg-f4xwd\""
Oct 30 22:17:15.374159 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:15.374124    4473 generic.go:356] "Generic (PLEG): container finished" podID="b78bb642-f809-4102-b4e5-ac485f1ca50d" containerID="ee015d5000a27c4f2af5043d5c85626434a172cf344e7389e85bfeb54dd244c0" exitCode=0
Oct 30 22:17:15.374358 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:15.374194    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w5-e431fc-74d48cd559-c7bht" event={"ID":"b78bb642-f809-4102-b4e5-ac485f1ca50d","Type":"ContainerDied","Data":"ee015d5000a27c4f2af5043d5c85626434a172cf344e7389e85bfeb54dd244c0"}
Oct 30 22:17:15.376156 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:15.376117    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cd728188-2246-4f1d-8f18-ec351a96b935-plugins-config-volume\") pod \"payload-processing-e2e-mt-242ddaf0-576ccc4d89-28sq2\" (UID: \"cd728188-2246-4f1d-8f18-ec351a96b935\") " pod="openshift-ingress/payload-processing-e2e-mt-242ddaf0-576ccc4d89-28sq2"
Oct 30 22:17:15.376331 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:15.376172    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v7vls\" (UniqueName: \"kubernetes.io/projected/cd728188-2246-4f1d-8f18-ec351a96b935-kube-api-access-v7vls\") pod \"payload-processing-e2e-mt-242ddaf0-576ccc4d89-28sq2\" (UID: \"cd728188-2246-4f1d-8f18-ec351a96b935\") " pod="openshift-ingress/payload-processing-e2e-mt-242ddaf0-576ccc4d89-28sq2"
Oct 30 22:17:15.377292 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:15.377268    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cd728188-2246-4f1d-8f18-ec351a96b935-plugins-config-volume\") pod \"payload-processing-e2e-mt-242ddaf0-576ccc4d89-28sq2\" (UID: \"cd728188-2246-4f1d-8f18-ec351a96b935\") " pod="openshift-ingress/payload-processing-e2e-mt-242ddaf0-576ccc4d89-28sq2"
Oct 30 22:17:15.435429 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:15.435396    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v7vls\" (UniqueName: \"kubernetes.io/projected/cd728188-2246-4f1d-8f18-ec351a96b935-kube-api-access-v7vls\") pod \"payload-processing-e2e-mt-242ddaf0-576ccc4d89-28sq2\" (UID: \"cd728188-2246-4f1d-8f18-ec351a96b935\") " pod="openshift-ingress/payload-processing-e2e-mt-242ddaf0-576ccc4d89-28sq2"
Oct 30 22:17:15.437431 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:15.437408    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w5-e431fc-74d48cd559-c7bht"
Oct 30 22:17:15.477720 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:15.477628    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b78bb642-f809-4102-b4e5-ac485f1ca50d-kube-api-access-r56kc\" (UniqueName: \"kubernetes.io/projected/b78bb642-f809-4102-b4e5-ac485f1ca50d-kube-api-access-r56kc\") pod \"b78bb642-f809-4102-b4e5-ac485f1ca50d\" (UID: \"b78bb642-f809-4102-b4e5-ac485f1ca50d\") "
Oct 30 22:17:15.477907 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:15.477815    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b78bb642-f809-4102-b4e5-ac485f1ca50d-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b78bb642-f809-4102-b4e5-ac485f1ca50d-plugins-config-volume\") pod \"b78bb642-f809-4102-b4e5-ac485f1ca50d\" (UID: \"b78bb642-f809-4102-b4e5-ac485f1ca50d\") "
Oct 30 22:17:15.478292 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:15.478265    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b78bb642-f809-4102-b4e5-ac485f1ca50d-plugins-config-volume" pod "b78bb642-f809-4102-b4e5-ac485f1ca50d" (UID: "b78bb642-f809-4102-b4e5-ac485f1ca50d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:17:15.480053 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:15.480033    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b78bb642-f809-4102-b4e5-ac485f1ca50d-kube-api-access-r56kc" pod "b78bb642-f809-4102-b4e5-ac485f1ca50d" (UID: "b78bb642-f809-4102-b4e5-ac485f1ca50d"). InnerVolumeSpecName "kube-api-access-r56kc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:17:15.583380 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:15.583339    4473 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r56kc\" (UniqueName: \"kubernetes.io/projected/b78bb642-f809-4102-b4e5-ac485f1ca50d-kube-api-access-r56kc\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:17:15.583380 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:15.583375    4473 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b78bb642-f809-4102-b4e5-ac485f1ca50d-plugins-config-volume\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:17:15.610348 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:15.610310    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-242ddaf0-576ccc4d89-28sq2"
Oct 30 22:17:15.669511 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:15.668825    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-242ddaf0-7b7b974bc-5rsm9"]
Oct 30 22:17:15.672103 ip-10-0-140-18 kubenswrapper[4473]: W1030 22:17:15.672065    4473 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod706bd83a_953d_4b2e_8f1c_e51bf4343a19.slice/crio-36c76f9df41456e4bae3c6301c96c882b80f8a2b00cf7b961c3a9e21ecb05597 WatchSource:0}: Error finding container 36c76f9df41456e4bae3c6301c96c882b80f8a2b00cf7b961c3a9e21ecb05597: Status 404 returned error can't find the container with id 36c76f9df41456e4bae3c6301c96c882b80f8a2b00cf7b961c3a9e21ecb05597
Oct 30 22:17:15.995974 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:15.995938    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-242ddaf0-576ccc4d89-28sq2"]
Oct 30 22:17:15.996695 ip-10-0-140-18 kubenswrapper[4473]: W1030 22:17:15.996665    4473 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd728188_2246_4f1d_8f18_ec351a96b935.slice/crio-4c3767f45501a73ce7c0125c76badb300c33fae7d6fb18d4d83ae3bf04580e33 WatchSource:0}: Error finding container 4c3767f45501a73ce7c0125c76badb300c33fae7d6fb18d4d83ae3bf04580e33: Status 404 returned error can't find the container with id 4c3767f45501a73ce7c0125c76badb300c33fae7d6fb18d4d83ae3bf04580e33
Oct 30 22:17:16.063225 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:16.063194    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-e431fc-5bb8bbd5f9-dgv2g"]
Oct 30 22:17:16.063434 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:16.063404    4473 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-e431fc-5bb8bbd5f9-dgv2g" podUID="806e3639-73c0-4dc2-87bf-4a4a95444e49" containerName="maas-api" containerID="cri-o://1c86b6a4c299895072f9e0fa08affdb99c0553a35b96a9364a9b4f069eaee262" gracePeriod=30
Oct 30 22:17:16.383536 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:16.383489    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-242ddaf0-576ccc4d89-28sq2" event={"ID":"cd728188-2246-4f1d-8f18-ec351a96b935","Type":"ContainerStarted","Data":"2aa491f0eb496338cbc817794a6e52e9bc7227a25da76367bff6045f176d81bb"}
Oct 30 22:17:16.383724 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:16.383561    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-242ddaf0-576ccc4d89-28sq2"
Oct 30 22:17:16.383724 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:16.383581    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-242ddaf0-576ccc4d89-28sq2" event={"ID":"cd728188-2246-4f1d-8f18-ec351a96b935","Type":"ContainerStarted","Data":"4c3767f45501a73ce7c0125c76badb300c33fae7d6fb18d4d83ae3bf04580e33"}
Oct 30 22:17:16.385201 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:16.385183    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w5-e431fc-74d48cd559-c7bht"
Oct 30 22:17:16.385308 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:16.385228    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w5-e431fc-74d48cd559-c7bht" event={"ID":"b78bb642-f809-4102-b4e5-ac485f1ca50d","Type":"ContainerDied","Data":"a3baf5317476b04f24191ae9448be093ff92bf87b9ede8ca8ea767dadd525aa3"}
Oct 30 22:17:16.385308 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:16.385282    4473 scope.go:122] "RemoveContainer" containerID="ee015d5000a27c4f2af5043d5c85626434a172cf344e7389e85bfeb54dd244c0"
Oct 30 22:17:16.387223 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:16.387197    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-242ddaf0-7b7b974bc-5rsm9" event={"ID":"706bd83a-953d-4b2e-8f1c-e51bf4343a19","Type":"ContainerStarted","Data":"79868169be9b695577a671c7b4eac2561bce79cfa7f25fbd94d5c89f4bb87cb0"}
Oct 30 22:17:16.387315 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:16.387240    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-242ddaf0-7b7b974bc-5rsm9" event={"ID":"706bd83a-953d-4b2e-8f1c-e51bf4343a19","Type":"ContainerStarted","Data":"36c76f9df41456e4bae3c6301c96c882b80f8a2b00cf7b961c3a9e21ecb05597"}
Oct 30 22:17:16.387860 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:16.387841    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-242ddaf0-7b7b974bc-5rsm9"
Oct 30 22:17:16.389372 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:16.389349    4473 generic.go:356] "Generic (PLEG): container finished" podID="806e3639-73c0-4dc2-87bf-4a4a95444e49" containerID="1c86b6a4c299895072f9e0fa08affdb99c0553a35b96a9364a9b4f069eaee262" exitCode=0
Oct 30 22:17:16.389455 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:16.389420    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-e431fc-5bb8bbd5f9-dgv2g" event={"ID":"806e3639-73c0-4dc2-87bf-4a4a95444e49","Type":"ContainerDied","Data":"1c86b6a4c299895072f9e0fa08affdb99c0553a35b96a9364a9b4f069eaee262"}
Oct 30 22:17:16.389506 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:16.389463    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-e431fc-5bb8bbd5f9-dgv2g" event={"ID":"806e3639-73c0-4dc2-87bf-4a4a95444e49","Type":"ContainerDied","Data":"0c6c26be27447a74301660dac06c853921d7391eda613cfdd32fe23335589485"}
Oct 30 22:17:16.389506 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:16.389482    4473 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0c6c26be27447a74301660dac06c853921d7391eda613cfdd32fe23335589485"
Oct 30 22:17:16.390556 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:16.390533    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-e431fc-5bb8bbd5f9-dgv2g"
Oct 30 22:17:16.497969 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:16.496575    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/806e3639-73c0-4dc2-87bf-4a4a95444e49-kube-api-access-n2clb\" (UniqueName: \"kubernetes.io/projected/806e3639-73c0-4dc2-87bf-4a4a95444e49-kube-api-access-n2clb\") pod \"806e3639-73c0-4dc2-87bf-4a4a95444e49\" (UID: \"806e3639-73c0-4dc2-87bf-4a4a95444e49\") "
Oct 30 22:17:16.497969 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:16.496710    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/806e3639-73c0-4dc2-87bf-4a4a95444e49-metrics-certs\" (UniqueName: \"kubernetes.io/secret/806e3639-73c0-4dc2-87bf-4a4a95444e49-metrics-certs\") pod \"806e3639-73c0-4dc2-87bf-4a4a95444e49\" (UID: \"806e3639-73c0-4dc2-87bf-4a4a95444e49\") "
Oct 30 22:17:16.497969 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:16.496770    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/806e3639-73c0-4dc2-87bf-4a4a95444e49-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/806e3639-73c0-4dc2-87bf-4a4a95444e49-openshift-service-ca\") pod \"806e3639-73c0-4dc2-87bf-4a4a95444e49\" (UID: \"806e3639-73c0-4dc2-87bf-4a4a95444e49\") "
Oct 30 22:17:16.497969 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:16.496946    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/806e3639-73c0-4dc2-87bf-4a4a95444e49-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/806e3639-73c0-4dc2-87bf-4a4a95444e49-maas-api-tls\") pod \"806e3639-73c0-4dc2-87bf-4a4a95444e49\" (UID: \"806e3639-73c0-4dc2-87bf-4a4a95444e49\") "
Oct 30 22:17:16.519204 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:16.519156    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/806e3639-73c0-4dc2-87bf-4a4a95444e49-openshift-service-ca" pod "806e3639-73c0-4dc2-87bf-4a4a95444e49" (UID: "806e3639-73c0-4dc2-87bf-4a4a95444e49"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:17:16.520596 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:16.520284    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/806e3639-73c0-4dc2-87bf-4a4a95444e49-metrics-certs" pod "806e3639-73c0-4dc2-87bf-4a4a95444e49" (UID: "806e3639-73c0-4dc2-87bf-4a4a95444e49"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:17:16.522365 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:16.522333    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/806e3639-73c0-4dc2-87bf-4a4a95444e49-kube-api-access-n2clb" pod "806e3639-73c0-4dc2-87bf-4a4a95444e49" (UID: "806e3639-73c0-4dc2-87bf-4a4a95444e49"). InnerVolumeSpecName "kube-api-access-n2clb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:17:16.529328 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:16.529263    4473 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-242ddaf0-576ccc4d89-28sq2" podStartSLOduration=1.5292382519999999 podStartE2EDuration="1.529238252s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:17:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:17:16.499354887 +0000 UTC m=+2082.272723387" watchObservedRunningTime="2026-10-30 22:17:16.529238252 +0000 UTC m=+2082.302606735"
Oct 30 22:17:16.529830 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:16.529793    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/806e3639-73c0-4dc2-87bf-4a4a95444e49-maas-api-tls" pod "806e3639-73c0-4dc2-87bf-4a4a95444e49" (UID: "806e3639-73c0-4dc2-87bf-4a4a95444e49"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:17:16.598847 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:16.598800    4473 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/806e3639-73c0-4dc2-87bf-4a4a95444e49-maas-api-tls\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:17:16.598847 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:16.598844    4473 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n2clb\" (UniqueName: \"kubernetes.io/projected/806e3639-73c0-4dc2-87bf-4a4a95444e49-kube-api-access-n2clb\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:17:16.598847 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:16.598858    4473 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/806e3639-73c0-4dc2-87bf-4a4a95444e49-metrics-certs\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:17:16.599121 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:16.598867    4473 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/806e3639-73c0-4dc2-87bf-4a4a95444e49-openshift-service-ca\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:17:16.737466 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:16.737412    4473 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-242ddaf0-7b7b974bc-5rsm9" podStartSLOduration=2.737395872 podStartE2EDuration="2.737395872s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:17:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:17:16.735486635 +0000 UTC m=+2082.508855126" watchObservedRunningTime="2026-10-30 22:17:16.737395872 +0000 UTC m=+2082.510764359"
Oct 30 22:17:16.900394 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:16.897709    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w5-e431fc-74d48cd559-c7bht"]
Oct 30 22:17:16.900394 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:16.897777    4473 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w5-e431fc-74d48cd559-c7bht"]
Oct 30 22:17:17.398806 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:17.398772    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-e431fc-5bb8bbd5f9-dgv2g"
Oct 30 22:17:17.567905 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:17.567872    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-e431fc-5bb8bbd5f9-dgv2g"]
Oct 30 22:17:17.618165 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:17.618130    4473 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-e431fc-5bb8bbd5f9-dgv2g"]
Oct 30 22:17:18.358972 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:18.358936    4473 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="806e3639-73c0-4dc2-87bf-4a4a95444e49" path="/var/lib/kubelet/pods/806e3639-73c0-4dc2-87bf-4a4a95444e49/volumes"
Oct 30 22:17:18.359389 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:18.359375    4473 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b78bb642-f809-4102-b4e5-ac485f1ca50d" path="/var/lib/kubelet/pods/b78bb642-f809-4102-b4e5-ac485f1ca50d/volumes"
Oct 30 22:17:18.847861 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:18.847827    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w0-ebddcc-openshift-default-5d98df957-nmwbj"]
Oct 30 22:17:18.848455 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:18.848106    4473 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w0-ebddcc-openshift-default-5d98df957-nmwbj" podUID="c15267dd-1a03-40d6-b6b1-ae0dec8cc68f" containerName="istio-proxy" containerID="cri-o://edc445cee72a17ddcf070a36329eb7577a159ae6d37e69d89ad5e4e189335ea3" gracePeriod=30
Oct 30 22:17:19.629679 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:19.629641    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-mg8sv"]
Oct 30 22:17:19.629901 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:19.629875    4473 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-mg8sv" podUID="117cd302-f5bc-4c57-9900-1b6ddb8c5e25" containerName="istio-proxy" containerID="cri-o://6b0dac1643a2721da8037f583e9119f35875e488b01803a244c4fc71eb136065" gracePeriod=30
Oct 30 22:17:19.657487 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:19.657450    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-vbr9j"]
Oct 30 22:17:19.657674 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:19.657653    4473 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-vbr9j" podUID="b9f9cb51-3fe0-43f3-9be3-95388ed892c8" containerName="istio-proxy" containerID="cri-o://29d33fff574cdf59f7fe25dbc0fc4a0badcfbe8a86d5cafaea44fea706d57817" gracePeriod=30
Oct 30 22:17:20.325980 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:17:20.325900    4473 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fncvg" containerName="extractor"
Oct 30 22:17:24.143350 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:24.143324    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w0-ebddcc-openshift-default-5d98df957-nmwbj"
Oct 30 22:17:24.231111 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:24.231077    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c15267dd-1a03-40d6-b6b1-ae0dec8cc68f-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c15267dd-1a03-40d6-b6b1-ae0dec8cc68f-workload-certs\") pod \"c15267dd-1a03-40d6-b6b1-ae0dec8cc68f\" (UID: \"c15267dd-1a03-40d6-b6b1-ae0dec8cc68f\") "
Oct 30 22:17:24.231295 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:24.231121    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c15267dd-1a03-40d6-b6b1-ae0dec8cc68f-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c15267dd-1a03-40d6-b6b1-ae0dec8cc68f-workload-socket\") pod \"c15267dd-1a03-40d6-b6b1-ae0dec8cc68f\" (UID: \"c15267dd-1a03-40d6-b6b1-ae0dec8cc68f\") "
Oct 30 22:17:24.231295 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:24.231169    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c15267dd-1a03-40d6-b6b1-ae0dec8cc68f-istio-token\" (UniqueName: \"kubernetes.io/projected/c15267dd-1a03-40d6-b6b1-ae0dec8cc68f-istio-token\") pod \"c15267dd-1a03-40d6-b6b1-ae0dec8cc68f\" (UID: \"c15267dd-1a03-40d6-b6b1-ae0dec8cc68f\") "
Oct 30 22:17:24.231295 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:24.231256    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c15267dd-1a03-40d6-b6b1-ae0dec8cc68f-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c15267dd-1a03-40d6-b6b1-ae0dec8cc68f-istio-envoy\") pod \"c15267dd-1a03-40d6-b6b1-ae0dec8cc68f\" (UID: \"c15267dd-1a03-40d6-b6b1-ae0dec8cc68f\") "
Oct 30 22:17:24.231464 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:24.231328    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/c15267dd-1a03-40d6-b6b1-ae0dec8cc68f-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c15267dd-1a03-40d6-b6b1-ae0dec8cc68f-istio-podinfo\") pod \"c15267dd-1a03-40d6-b6b1-ae0dec8cc68f\" (UID: \"c15267dd-1a03-40d6-b6b1-ae0dec8cc68f\") "
Oct 30 22:17:24.231464 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:24.231358    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c15267dd-1a03-40d6-b6b1-ae0dec8cc68f-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c15267dd-1a03-40d6-b6b1-ae0dec8cc68f-credential-socket\") pod \"c15267dd-1a03-40d6-b6b1-ae0dec8cc68f\" (UID: \"c15267dd-1a03-40d6-b6b1-ae0dec8cc68f\") "
Oct 30 22:17:24.231464 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:24.231386    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c15267dd-1a03-40d6-b6b1-ae0dec8cc68f-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c15267dd-1a03-40d6-b6b1-ae0dec8cc68f-istio-data\") pod \"c15267dd-1a03-40d6-b6b1-ae0dec8cc68f\" (UID: \"c15267dd-1a03-40d6-b6b1-ae0dec8cc68f\") "
Oct 30 22:17:24.231464 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:24.231403    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c15267dd-1a03-40d6-b6b1-ae0dec8cc68f-workload-certs" pod "c15267dd-1a03-40d6-b6b1-ae0dec8cc68f" (UID: "c15267dd-1a03-40d6-b6b1-ae0dec8cc68f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:17:24.231464 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:24.231421    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c15267dd-1a03-40d6-b6b1-ae0dec8cc68f-kube-api-access-jnzmz\" (UniqueName: \"kubernetes.io/projected/c15267dd-1a03-40d6-b6b1-ae0dec8cc68f-kube-api-access-jnzmz\") pod \"c15267dd-1a03-40d6-b6b1-ae0dec8cc68f\" (UID: \"c15267dd-1a03-40d6-b6b1-ae0dec8cc68f\") "
Oct 30 22:17:24.231734 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:24.231499    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c15267dd-1a03-40d6-b6b1-ae0dec8cc68f-workload-socket" pod "c15267dd-1a03-40d6-b6b1-ae0dec8cc68f" (UID: "c15267dd-1a03-40d6-b6b1-ae0dec8cc68f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:17:24.231734 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:24.231582    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c15267dd-1a03-40d6-b6b1-ae0dec8cc68f-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c15267dd-1a03-40d6-b6b1-ae0dec8cc68f-istiod-ca-cert\") pod \"c15267dd-1a03-40d6-b6b1-ae0dec8cc68f\" (UID: \"c15267dd-1a03-40d6-b6b1-ae0dec8cc68f\") "
Oct 30 22:17:24.231734 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:24.231648    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c15267dd-1a03-40d6-b6b1-ae0dec8cc68f-credential-socket" pod "c15267dd-1a03-40d6-b6b1-ae0dec8cc68f" (UID: "c15267dd-1a03-40d6-b6b1-ae0dec8cc68f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:17:24.231734 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:24.231706    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c15267dd-1a03-40d6-b6b1-ae0dec8cc68f-istio-data" pod "c15267dd-1a03-40d6-b6b1-ae0dec8cc68f" (UID: "c15267dd-1a03-40d6-b6b1-ae0dec8cc68f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:17:24.232109 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:24.232086    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c15267dd-1a03-40d6-b6b1-ae0dec8cc68f-istiod-ca-cert" pod "c15267dd-1a03-40d6-b6b1-ae0dec8cc68f" (UID: "c15267dd-1a03-40d6-b6b1-ae0dec8cc68f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:17:24.232219 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:24.232201    4473 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c15267dd-1a03-40d6-b6b1-ae0dec8cc68f-istiod-ca-cert\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:17:24.232296 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:24.232225    4473 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c15267dd-1a03-40d6-b6b1-ae0dec8cc68f-workload-certs\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:17:24.232296 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:24.232238    4473 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c15267dd-1a03-40d6-b6b1-ae0dec8cc68f-workload-socket\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:17:24.232296 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:24.232250    4473 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c15267dd-1a03-40d6-b6b1-ae0dec8cc68f-credential-socket\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:17:24.232296 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:24.232263    4473 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c15267dd-1a03-40d6-b6b1-ae0dec8cc68f-istio-data\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:17:24.234186 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:24.234145    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c15267dd-1a03-40d6-b6b1-ae0dec8cc68f-istio-podinfo" pod "c15267dd-1a03-40d6-b6b1-ae0dec8cc68f" (UID: "c15267dd-1a03-40d6-b6b1-ae0dec8cc68f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:17:24.234558 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:24.234533    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c15267dd-1a03-40d6-b6b1-ae0dec8cc68f-kube-api-access-jnzmz" pod "c15267dd-1a03-40d6-b6b1-ae0dec8cc68f" (UID: "c15267dd-1a03-40d6-b6b1-ae0dec8cc68f"). InnerVolumeSpecName "kube-api-access-jnzmz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:17:24.234749 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:24.234728    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c15267dd-1a03-40d6-b6b1-ae0dec8cc68f-istio-token" pod "c15267dd-1a03-40d6-b6b1-ae0dec8cc68f" (UID: "c15267dd-1a03-40d6-b6b1-ae0dec8cc68f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:17:24.234860 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:24.234790    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c15267dd-1a03-40d6-b6b1-ae0dec8cc68f-istio-envoy" pod "c15267dd-1a03-40d6-b6b1-ae0dec8cc68f" (UID: "c15267dd-1a03-40d6-b6b1-ae0dec8cc68f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:17:24.333186 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:24.333154    4473 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c15267dd-1a03-40d6-b6b1-ae0dec8cc68f-istio-envoy\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:17:24.333186 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:24.333183    4473 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c15267dd-1a03-40d6-b6b1-ae0dec8cc68f-istio-podinfo\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:17:24.333186 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:24.333194    4473 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jnzmz\" (UniqueName: \"kubernetes.io/projected/c15267dd-1a03-40d6-b6b1-ae0dec8cc68f-kube-api-access-jnzmz\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:17:24.333186 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:24.333205    4473 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c15267dd-1a03-40d6-b6b1-ae0dec8cc68f-istio-token\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:17:24.450383 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:24.450301    4473 generic.go:356] "Generic (PLEG): container finished" podID="c15267dd-1a03-40d6-b6b1-ae0dec8cc68f" containerID="edc445cee72a17ddcf070a36329eb7577a159ae6d37e69d89ad5e4e189335ea3" exitCode=0
Oct 30 22:17:24.450383 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:24.450340    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w0-ebddcc-openshift-default-5d98df957-nmwbj" event={"ID":"c15267dd-1a03-40d6-b6b1-ae0dec8cc68f","Type":"ContainerDied","Data":"edc445cee72a17ddcf070a36329eb7577a159ae6d37e69d89ad5e4e189335ea3"}
Oct 30 22:17:24.450576 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:24.450400    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w0-ebddcc-openshift-default-5d98df957-nmwbj" event={"ID":"c15267dd-1a03-40d6-b6b1-ae0dec8cc68f","Type":"ContainerDied","Data":"6282fdf29d19a4135c8c4f29ec1342238a88e6b5d17863cf357cc375256a7885"}
Oct 30 22:17:24.450576 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:24.450412    4473 scope.go:122] "RemoveContainer" containerID="edc445cee72a17ddcf070a36329eb7577a159ae6d37e69d89ad5e4e189335ea3"
Oct 30 22:17:24.450576 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:24.450355    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w0-ebddcc-openshift-default-5d98df957-nmwbj"
Oct 30 22:17:24.460952 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:24.460903    4473 scope.go:122] "RemoveContainer" containerID="39290de07fb2eaf514609b7a187a6af002d0fbd2a976ebcda2ba8f7cf3298592"
Oct 30 22:17:24.471553 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:24.471525    4473 scope.go:122] "RemoveContainer" containerID="edc445cee72a17ddcf070a36329eb7577a159ae6d37e69d89ad5e4e189335ea3"
Oct 30 22:17:24.471887 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:17:24.471866    4473 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"edc445cee72a17ddcf070a36329eb7577a159ae6d37e69d89ad5e4e189335ea3\": container with ID starting with edc445cee72a17ddcf070a36329eb7577a159ae6d37e69d89ad5e4e189335ea3 not found: ID does not exist" containerID="edc445cee72a17ddcf070a36329eb7577a159ae6d37e69d89ad5e4e189335ea3"
Oct 30 22:17:24.472026 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:24.471896    4473 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"edc445cee72a17ddcf070a36329eb7577a159ae6d37e69d89ad5e4e189335ea3"} err="failed to get container status \"edc445cee72a17ddcf070a36329eb7577a159ae6d37e69d89ad5e4e189335ea3\": rpc error: code = NotFound desc = could not find container \"edc445cee72a17ddcf070a36329eb7577a159ae6d37e69d89ad5e4e189335ea3\": container with ID starting with edc445cee72a17ddcf070a36329eb7577a159ae6d37e69d89ad5e4e189335ea3 not found: ID does not exist"
Oct 30 22:17:24.472026 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:24.471931    4473 scope.go:122] "RemoveContainer" containerID="39290de07fb2eaf514609b7a187a6af002d0fbd2a976ebcda2ba8f7cf3298592"
Oct 30 22:17:24.472268 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:17:24.472244    4473 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"39290de07fb2eaf514609b7a187a6af002d0fbd2a976ebcda2ba8f7cf3298592\": container with ID starting with 39290de07fb2eaf514609b7a187a6af002d0fbd2a976ebcda2ba8f7cf3298592 not found: ID does not exist" containerID="39290de07fb2eaf514609b7a187a6af002d0fbd2a976ebcda2ba8f7cf3298592"
Oct 30 22:17:24.472322 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:24.472277    4473 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"39290de07fb2eaf514609b7a187a6af002d0fbd2a976ebcda2ba8f7cf3298592"} err="failed to get container status \"39290de07fb2eaf514609b7a187a6af002d0fbd2a976ebcda2ba8f7cf3298592\": rpc error: code = NotFound desc = could not find container \"39290de07fb2eaf514609b7a187a6af002d0fbd2a976ebcda2ba8f7cf3298592\": container with ID starting with 39290de07fb2eaf514609b7a187a6af002d0fbd2a976ebcda2ba8f7cf3298592 not found: ID does not exist"
Oct 30 22:17:24.620982 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:24.620948    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w0-ebddcc-openshift-default-5d98df957-nmwbj"]
Oct 30 22:17:24.623622 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:24.623599    4473 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w0-ebddcc-openshift-default-5d98df957-nmwbj"]
Oct 30 22:17:24.723976 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:17:24.723902    4473 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod117cd302_f5bc_4c57_9900_1b6ddb8c5e25.slice/crio-conmon-6b0dac1643a2721da8037f583e9119f35875e488b01803a244c4fc71eb136065.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod117cd302_f5bc_4c57_9900_1b6ddb8c5e25.slice/crio-6b0dac1643a2721da8037f583e9119f35875e488b01803a244c4fc71eb136065.scope\": RecentStats: unable to find data in memory cache]"
Oct 30 22:17:24.884945 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:24.880886    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w5-e431fc-openshift-default-6d78d86df7-lhwsl"]
Oct 30 22:17:24.885804 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:24.885764    4473 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w5-e431fc-openshift-default-6d78d86df7-lhwsl" podUID="61782e91-1520-4b83-ad8d-8dd944881b52" containerName="istio-proxy" containerID="cri-o://507749823c9e799137e00bbb0c980cdf0023eb744fa7b5df0da742998c5ddc81" gracePeriod=30
Oct 30 22:17:24.953387 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:24.953357    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-vbr9j"
Oct 30 22:17:25.054537 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:25.054447    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b9f9cb51-3fe0-43f3-9be3-95388ed892c8-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b9f9cb51-3fe0-43f3-9be3-95388ed892c8-istio-data\") pod \"b9f9cb51-3fe0-43f3-9be3-95388ed892c8\" (UID: \"b9f9cb51-3fe0-43f3-9be3-95388ed892c8\") "
Oct 30 22:17:25.054849 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:25.054816    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b9f9cb51-3fe0-43f3-9be3-95388ed892c8-kube-api-access-r2gjp\" (UniqueName: \"kubernetes.io/projected/b9f9cb51-3fe0-43f3-9be3-95388ed892c8-kube-api-access-r2gjp\") pod \"b9f9cb51-3fe0-43f3-9be3-95388ed892c8\" (UID: \"b9f9cb51-3fe0-43f3-9be3-95388ed892c8\") "
Oct 30 22:17:25.055142 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:25.055104    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/b9f9cb51-3fe0-43f3-9be3-95388ed892c8-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b9f9cb51-3fe0-43f3-9be3-95388ed892c8-istio-podinfo\") pod \"b9f9cb51-3fe0-43f3-9be3-95388ed892c8\" (UID: \"b9f9cb51-3fe0-43f3-9be3-95388ed892c8\") "
Oct 30 22:17:25.055265 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:25.055246    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b9f9cb51-3fe0-43f3-9be3-95388ed892c8-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b9f9cb51-3fe0-43f3-9be3-95388ed892c8-istio-envoy\") pod \"b9f9cb51-3fe0-43f3-9be3-95388ed892c8\" (UID: \"b9f9cb51-3fe0-43f3-9be3-95388ed892c8\") "
Oct 30 22:17:25.055318 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:25.055299    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b9f9cb51-3fe0-43f3-9be3-95388ed892c8-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b9f9cb51-3fe0-43f3-9be3-95388ed892c8-credential-socket\") pod \"b9f9cb51-3fe0-43f3-9be3-95388ed892c8\" (UID: \"b9f9cb51-3fe0-43f3-9be3-95388ed892c8\") "
Oct 30 22:17:25.055382 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:25.055354    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b9f9cb51-3fe0-43f3-9be3-95388ed892c8-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b9f9cb51-3fe0-43f3-9be3-95388ed892c8-workload-socket\") pod \"b9f9cb51-3fe0-43f3-9be3-95388ed892c8\" (UID: \"b9f9cb51-3fe0-43f3-9be3-95388ed892c8\") "
Oct 30 22:17:25.055463 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:25.055392    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b9f9cb51-3fe0-43f3-9be3-95388ed892c8-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b9f9cb51-3fe0-43f3-9be3-95388ed892c8-istiod-ca-cert\") pod \"b9f9cb51-3fe0-43f3-9be3-95388ed892c8\" (UID: \"b9f9cb51-3fe0-43f3-9be3-95388ed892c8\") "
Oct 30 22:17:25.055463 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:25.055436    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b9f9cb51-3fe0-43f3-9be3-95388ed892c8-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b9f9cb51-3fe0-43f3-9be3-95388ed892c8-workload-certs\") pod \"b9f9cb51-3fe0-43f3-9be3-95388ed892c8\" (UID: \"b9f9cb51-3fe0-43f3-9be3-95388ed892c8\") "
Oct 30 22:17:25.055738 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:25.055535    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b9f9cb51-3fe0-43f3-9be3-95388ed892c8-istio-token\" (UniqueName: \"kubernetes.io/projected/b9f9cb51-3fe0-43f3-9be3-95388ed892c8-istio-token\") pod \"b9f9cb51-3fe0-43f3-9be3-95388ed892c8\" (UID: \"b9f9cb51-3fe0-43f3-9be3-95388ed892c8\") "
Oct 30 22:17:25.056552 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:25.054977    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b9f9cb51-3fe0-43f3-9be3-95388ed892c8-istio-data" pod "b9f9cb51-3fe0-43f3-9be3-95388ed892c8" (UID: "b9f9cb51-3fe0-43f3-9be3-95388ed892c8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:17:25.056655 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:25.056637    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b9f9cb51-3fe0-43f3-9be3-95388ed892c8-workload-socket" pod "b9f9cb51-3fe0-43f3-9be3-95388ed892c8" (UID: "b9f9cb51-3fe0-43f3-9be3-95388ed892c8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:17:25.056979 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:25.056914    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b9f9cb51-3fe0-43f3-9be3-95388ed892c8-credential-socket" pod "b9f9cb51-3fe0-43f3-9be3-95388ed892c8" (UID: "b9f9cb51-3fe0-43f3-9be3-95388ed892c8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:17:25.057289 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:25.057167    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b9f9cb51-3fe0-43f3-9be3-95388ed892c8-workload-certs" pod "b9f9cb51-3fe0-43f3-9be3-95388ed892c8" (UID: "b9f9cb51-3fe0-43f3-9be3-95388ed892c8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:17:25.057289 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:25.057239    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b9f9cb51-3fe0-43f3-9be3-95388ed892c8-istiod-ca-cert" pod "b9f9cb51-3fe0-43f3-9be3-95388ed892c8" (UID: "b9f9cb51-3fe0-43f3-9be3-95388ed892c8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:17:25.057811 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:25.057784    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b9f9cb51-3fe0-43f3-9be3-95388ed892c8-kube-api-access-r2gjp" pod "b9f9cb51-3fe0-43f3-9be3-95388ed892c8" (UID: "b9f9cb51-3fe0-43f3-9be3-95388ed892c8"). InnerVolumeSpecName "kube-api-access-r2gjp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:17:25.060384 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:25.060360    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b9f9cb51-3fe0-43f3-9be3-95388ed892c8-istio-token" pod "b9f9cb51-3fe0-43f3-9be3-95388ed892c8" (UID: "b9f9cb51-3fe0-43f3-9be3-95388ed892c8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:17:25.062387 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:25.062361    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b9f9cb51-3fe0-43f3-9be3-95388ed892c8-istio-podinfo" pod "b9f9cb51-3fe0-43f3-9be3-95388ed892c8" (UID: "b9f9cb51-3fe0-43f3-9be3-95388ed892c8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:17:25.063157 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:25.063135    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b9f9cb51-3fe0-43f3-9be3-95388ed892c8-istio-envoy" pod "b9f9cb51-3fe0-43f3-9be3-95388ed892c8" (UID: "b9f9cb51-3fe0-43f3-9be3-95388ed892c8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:17:25.157772 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:25.157733    4473 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b9f9cb51-3fe0-43f3-9be3-95388ed892c8-istiod-ca-cert\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:17:25.157772 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:25.157765    4473 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b9f9cb51-3fe0-43f3-9be3-95388ed892c8-workload-certs\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:17:25.157772 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:25.157776    4473 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b9f9cb51-3fe0-43f3-9be3-95388ed892c8-istio-token\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:17:25.158279 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:25.157785    4473 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b9f9cb51-3fe0-43f3-9be3-95388ed892c8-istio-data\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:17:25.158279 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:25.157795    4473 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r2gjp\" (UniqueName: \"kubernetes.io/projected/b9f9cb51-3fe0-43f3-9be3-95388ed892c8-kube-api-access-r2gjp\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:17:25.158279 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:25.157805    4473 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b9f9cb51-3fe0-43f3-9be3-95388ed892c8-istio-podinfo\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:17:25.158279 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:25.157813    4473 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b9f9cb51-3fe0-43f3-9be3-95388ed892c8-istio-envoy\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:17:25.158279 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:25.157821    4473 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b9f9cb51-3fe0-43f3-9be3-95388ed892c8-credential-socket\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:17:25.158279 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:25.157831    4473 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b9f9cb51-3fe0-43f3-9be3-95388ed892c8-workload-socket\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:17:25.311313 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:25.311252    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-mg8sv"
Oct 30 22:17:25.391619 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:25.391584    4473 patch_prober.go:29] interesting pod/e2e-shared-a-w5-e431fc-openshift-default-6d78d86df7-lhwsl container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 22:17:25.391791 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:25.391641    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w5-e431fc-openshift-default-6d78d86df7-lhwsl" podUID="61782e91-1520-4b83-ad8d-8dd944881b52" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 22:17:25.460838 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:25.460798    4473 generic.go:356] "Generic (PLEG): container finished" podID="117cd302-f5bc-4c57-9900-1b6ddb8c5e25" containerID="6b0dac1643a2721da8037f583e9119f35875e488b01803a244c4fc71eb136065" exitCode=0
Oct 30 22:17:25.461022 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:25.460851    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-mg8sv"
Oct 30 22:17:25.461022 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:25.460883    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-mg8sv" event={"ID":"117cd302-f5bc-4c57-9900-1b6ddb8c5e25","Type":"ContainerDied","Data":"6b0dac1643a2721da8037f583e9119f35875e488b01803a244c4fc71eb136065"}
Oct 30 22:17:25.461022 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:25.460956    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-mg8sv" event={"ID":"117cd302-f5bc-4c57-9900-1b6ddb8c5e25","Type":"ContainerDied","Data":"ff51301e3d8951109a7ec2dcbf410000acafe044654bfa7253e6bd11ca8a5b64"}
Oct 30 22:17:25.461022 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:25.460981    4473 scope.go:122] "RemoveContainer" containerID="6b0dac1643a2721da8037f583e9119f35875e488b01803a244c4fc71eb136065"
Oct 30 22:17:25.461412 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:25.461390    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/117cd302-f5bc-4c57-9900-1b6ddb8c5e25-istio-token\" (UniqueName: \"kubernetes.io/projected/117cd302-f5bc-4c57-9900-1b6ddb8c5e25-istio-token\") pod \"117cd302-f5bc-4c57-9900-1b6ddb8c5e25\" (UID: \"117cd302-f5bc-4c57-9900-1b6ddb8c5e25\") "
Oct 30 22:17:25.461472 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:25.461442    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/117cd302-f5bc-4c57-9900-1b6ddb8c5e25-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/117cd302-f5bc-4c57-9900-1b6ddb8c5e25-workload-certs\") pod \"117cd302-f5bc-4c57-9900-1b6ddb8c5e25\" (UID: \"117cd302-f5bc-4c57-9900-1b6ddb8c5e25\") "
Oct 30 22:17:25.461522 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:25.461492    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/117cd302-f5bc-4c57-9900-1b6ddb8c5e25-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/117cd302-f5bc-4c57-9900-1b6ddb8c5e25-istio-data\") pod \"117cd302-f5bc-4c57-9900-1b6ddb8c5e25\" (UID: \"117cd302-f5bc-4c57-9900-1b6ddb8c5e25\") "
Oct 30 22:17:25.461668 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:25.461651    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/117cd302-f5bc-4c57-9900-1b6ddb8c5e25-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/117cd302-f5bc-4c57-9900-1b6ddb8c5e25-istio-envoy\") pod \"117cd302-f5bc-4c57-9900-1b6ddb8c5e25\" (UID: \"117cd302-f5bc-4c57-9900-1b6ddb8c5e25\") "
Oct 30 22:17:25.461765 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:25.461728    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/117cd302-f5bc-4c57-9900-1b6ddb8c5e25-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/117cd302-f5bc-4c57-9900-1b6ddb8c5e25-credential-socket\") pod \"117cd302-f5bc-4c57-9900-1b6ddb8c5e25\" (UID: \"117cd302-f5bc-4c57-9900-1b6ddb8c5e25\") "
Oct 30 22:17:25.461822 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:25.461793    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/117cd302-f5bc-4c57-9900-1b6ddb8c5e25-workload-certs" pod "117cd302-f5bc-4c57-9900-1b6ddb8c5e25" (UID: "117cd302-f5bc-4c57-9900-1b6ddb8c5e25"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:17:25.461877 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:25.461833    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/117cd302-f5bc-4c57-9900-1b6ddb8c5e25-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/117cd302-f5bc-4c57-9900-1b6ddb8c5e25-workload-socket\") pod \"117cd302-f5bc-4c57-9900-1b6ddb8c5e25\" (UID: \"117cd302-f5bc-4c57-9900-1b6ddb8c5e25\") "
Oct 30 22:17:25.461877 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:25.461865    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/117cd302-f5bc-4c57-9900-1b6ddb8c5e25-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/117cd302-f5bc-4c57-9900-1b6ddb8c5e25-istiod-ca-cert\") pod \"117cd302-f5bc-4c57-9900-1b6ddb8c5e25\" (UID: \"117cd302-f5bc-4c57-9900-1b6ddb8c5e25\") "
Oct 30 22:17:25.462022 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:25.461900    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/117cd302-f5bc-4c57-9900-1b6ddb8c5e25-istio-data" pod "117cd302-f5bc-4c57-9900-1b6ddb8c5e25" (UID: "117cd302-f5bc-4c57-9900-1b6ddb8c5e25"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:17:25.462022 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:25.461904    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/117cd302-f5bc-4c57-9900-1b6ddb8c5e25-kube-api-access-szrl4\" (UniqueName: \"kubernetes.io/projected/117cd302-f5bc-4c57-9900-1b6ddb8c5e25-kube-api-access-szrl4\") pod \"117cd302-f5bc-4c57-9900-1b6ddb8c5e25\" (UID: \"117cd302-f5bc-4c57-9900-1b6ddb8c5e25\") "
Oct 30 22:17:25.462130 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:25.462017    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/117cd302-f5bc-4c57-9900-1b6ddb8c5e25-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/117cd302-f5bc-4c57-9900-1b6ddb8c5e25-istio-podinfo\") pod \"117cd302-f5bc-4c57-9900-1b6ddb8c5e25\" (UID: \"117cd302-f5bc-4c57-9900-1b6ddb8c5e25\") "
Oct 30 22:17:25.462130 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:25.462089    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/117cd302-f5bc-4c57-9900-1b6ddb8c5e25-credential-socket" pod "117cd302-f5bc-4c57-9900-1b6ddb8c5e25" (UID: "117cd302-f5bc-4c57-9900-1b6ddb8c5e25"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:17:25.462315 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:25.462290    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/117cd302-f5bc-4c57-9900-1b6ddb8c5e25-workload-socket" pod "117cd302-f5bc-4c57-9900-1b6ddb8c5e25" (UID: "117cd302-f5bc-4c57-9900-1b6ddb8c5e25"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:17:25.462844 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:25.462795    4473 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/117cd302-f5bc-4c57-9900-1b6ddb8c5e25-workload-socket\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:17:25.462844 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:25.462820    4473 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/117cd302-f5bc-4c57-9900-1b6ddb8c5e25-workload-certs\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:17:25.463328 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:25.462859    4473 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/117cd302-f5bc-4c57-9900-1b6ddb8c5e25-istio-data\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:17:25.463328 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:25.462875    4473 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/117cd302-f5bc-4c57-9900-1b6ddb8c5e25-credential-socket\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:17:25.463328 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:25.462794    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/117cd302-f5bc-4c57-9900-1b6ddb8c5e25-istiod-ca-cert" pod "117cd302-f5bc-4c57-9900-1b6ddb8c5e25" (UID: "117cd302-f5bc-4c57-9900-1b6ddb8c5e25"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:17:25.464520 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:25.464494    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/117cd302-f5bc-4c57-9900-1b6ddb8c5e25-kube-api-access-szrl4" pod "117cd302-f5bc-4c57-9900-1b6ddb8c5e25" (UID: "117cd302-f5bc-4c57-9900-1b6ddb8c5e25"). InnerVolumeSpecName "kube-api-access-szrl4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:17:25.464618 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:25.464533    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/117cd302-f5bc-4c57-9900-1b6ddb8c5e25-istio-envoy" pod "117cd302-f5bc-4c57-9900-1b6ddb8c5e25" (UID: "117cd302-f5bc-4c57-9900-1b6ddb8c5e25"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:17:25.464781 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:25.464758    4473 generic.go:356] "Generic (PLEG): container finished" podID="b9f9cb51-3fe0-43f3-9be3-95388ed892c8" containerID="29d33fff574cdf59f7fe25dbc0fc4a0badcfbe8a86d5cafaea44fea706d57817" exitCode=0
Oct 30 22:17:25.464977 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:25.464831    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-vbr9j"
Oct 30 22:17:25.465091 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:25.464962    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-vbr9j" event={"ID":"b9f9cb51-3fe0-43f3-9be3-95388ed892c8","Type":"ContainerDied","Data":"29d33fff574cdf59f7fe25dbc0fc4a0badcfbe8a86d5cafaea44fea706d57817"}
Oct 30 22:17:25.465091 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:25.465032    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-vbr9j" event={"ID":"b9f9cb51-3fe0-43f3-9be3-95388ed892c8","Type":"ContainerDied","Data":"e318c54d504541c8b537b6697d3fb2f924bfc6a0ec9ee0cc30f673f4451915b4"}
Oct 30 22:17:25.465289 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:25.464940    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/117cd302-f5bc-4c57-9900-1b6ddb8c5e25-istio-podinfo" pod "117cd302-f5bc-4c57-9900-1b6ddb8c5e25" (UID: "117cd302-f5bc-4c57-9900-1b6ddb8c5e25"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:17:25.466611 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:25.466591    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/117cd302-f5bc-4c57-9900-1b6ddb8c5e25-istio-token" pod "117cd302-f5bc-4c57-9900-1b6ddb8c5e25" (UID: "117cd302-f5bc-4c57-9900-1b6ddb8c5e25"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:17:25.475762 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:25.475732    4473 scope.go:122] "RemoveContainer" containerID="6b0dac1643a2721da8037f583e9119f35875e488b01803a244c4fc71eb136065"
Oct 30 22:17:25.476099 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:17:25.476079    4473 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6b0dac1643a2721da8037f583e9119f35875e488b01803a244c4fc71eb136065\": container with ID starting with 6b0dac1643a2721da8037f583e9119f35875e488b01803a244c4fc71eb136065 not found: ID does not exist" containerID="6b0dac1643a2721da8037f583e9119f35875e488b01803a244c4fc71eb136065"
Oct 30 22:17:25.476178 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:25.476107    4473 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6b0dac1643a2721da8037f583e9119f35875e488b01803a244c4fc71eb136065"} err="failed to get container status \"6b0dac1643a2721da8037f583e9119f35875e488b01803a244c4fc71eb136065\": rpc error: code = NotFound desc = could not find container \"6b0dac1643a2721da8037f583e9119f35875e488b01803a244c4fc71eb136065\": container with ID starting with 6b0dac1643a2721da8037f583e9119f35875e488b01803a244c4fc71eb136065 not found: ID does not exist"
Oct 30 22:17:25.476178 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:25.476124    4473 scope.go:122] "RemoveContainer" containerID="29d33fff574cdf59f7fe25dbc0fc4a0badcfbe8a86d5cafaea44fea706d57817"
Oct 30 22:17:25.485320 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:25.485301    4473 scope.go:122] "RemoveContainer" containerID="29d33fff574cdf59f7fe25dbc0fc4a0badcfbe8a86d5cafaea44fea706d57817"
Oct 30 22:17:25.485588 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:17:25.485565    4473 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"29d33fff574cdf59f7fe25dbc0fc4a0badcfbe8a86d5cafaea44fea706d57817\": container with ID starting with 29d33fff574cdf59f7fe25dbc0fc4a0badcfbe8a86d5cafaea44fea706d57817 not found: ID does not exist" containerID="29d33fff574cdf59f7fe25dbc0fc4a0badcfbe8a86d5cafaea44fea706d57817"
Oct 30 22:17:25.485661 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:25.485596    4473 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"29d33fff574cdf59f7fe25dbc0fc4a0badcfbe8a86d5cafaea44fea706d57817"} err="failed to get container status \"29d33fff574cdf59f7fe25dbc0fc4a0badcfbe8a86d5cafaea44fea706d57817\": rpc error: code = NotFound desc = could not find container \"29d33fff574cdf59f7fe25dbc0fc4a0badcfbe8a86d5cafaea44fea706d57817\": container with ID starting with 29d33fff574cdf59f7fe25dbc0fc4a0badcfbe8a86d5cafaea44fea706d57817 not found: ID does not exist"
Oct 30 22:17:25.564466 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:25.564390    4473 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/117cd302-f5bc-4c57-9900-1b6ddb8c5e25-istiod-ca-cert\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:17:25.564466 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:25.564420    4473 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-szrl4\" (UniqueName: \"kubernetes.io/projected/117cd302-f5bc-4c57-9900-1b6ddb8c5e25-kube-api-access-szrl4\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:17:25.564466 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:25.564430    4473 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/117cd302-f5bc-4c57-9900-1b6ddb8c5e25-istio-podinfo\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:17:25.564466 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:25.564439    4473 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/117cd302-f5bc-4c57-9900-1b6ddb8c5e25-istio-token\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:17:25.564466 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:25.564447    4473 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/117cd302-f5bc-4c57-9900-1b6ddb8c5e25-istio-envoy\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:17:25.634977 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:25.634937    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-vbr9j"]
Oct 30 22:17:25.637768 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:25.637737    4473 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-vbr9j"]
Oct 30 22:17:25.883642 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:25.883612    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-mg8sv"]
Oct 30 22:17:25.886561 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:25.886539    4473 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-b0c55f-openshift-default-85f66ff7f5-mg8sv"]
Oct 30 22:17:26.358964 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:26.358872    4473 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="117cd302-f5bc-4c57-9900-1b6ddb8c5e25" path="/var/lib/kubelet/pods/117cd302-f5bc-4c57-9900-1b6ddb8c5e25/volumes"
Oct 30 22:17:26.359429 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:26.359413    4473 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b9f9cb51-3fe0-43f3-9be3-95388ed892c8" path="/var/lib/kubelet/pods/b9f9cb51-3fe0-43f3-9be3-95388ed892c8/volumes"
Oct 30 22:17:26.359825 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:26.359812    4473 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c15267dd-1a03-40d6-b6b1-ae0dec8cc68f" path="/var/lib/kubelet/pods/c15267dd-1a03-40d6-b6b1-ae0dec8cc68f/volumes"
Oct 30 22:17:27.402598 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:27.402566    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-242ddaf0-576ccc4d89-28sq2"
Oct 30 22:17:28.412314 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:28.412281    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-242ddaf0-7b7b974bc-5rsm9"
Oct 30 22:17:30.149970 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:30.149942    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w5-e431fc-openshift-default-6d78d86df7-lhwsl"
Oct 30 22:17:30.226441 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:30.226346    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/61782e91-1520-4b83-ad8d-8dd944881b52-istio-token\" (UniqueName: \"kubernetes.io/projected/61782e91-1520-4b83-ad8d-8dd944881b52-istio-token\") pod \"61782e91-1520-4b83-ad8d-8dd944881b52\" (UID: \"61782e91-1520-4b83-ad8d-8dd944881b52\") "
Oct 30 22:17:30.226441 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:30.226416    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/61782e91-1520-4b83-ad8d-8dd944881b52-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/61782e91-1520-4b83-ad8d-8dd944881b52-istio-envoy\") pod \"61782e91-1520-4b83-ad8d-8dd944881b52\" (UID: \"61782e91-1520-4b83-ad8d-8dd944881b52\") "
Oct 30 22:17:30.226441 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:30.226439    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/61782e91-1520-4b83-ad8d-8dd944881b52-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/61782e91-1520-4b83-ad8d-8dd944881b52-istiod-ca-cert\") pod \"61782e91-1520-4b83-ad8d-8dd944881b52\" (UID: \"61782e91-1520-4b83-ad8d-8dd944881b52\") "
Oct 30 22:17:30.226706 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:30.226480    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/61782e91-1520-4b83-ad8d-8dd944881b52-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/61782e91-1520-4b83-ad8d-8dd944881b52-workload-socket\") pod \"61782e91-1520-4b83-ad8d-8dd944881b52\" (UID: \"61782e91-1520-4b83-ad8d-8dd944881b52\") "
Oct 30 22:17:30.226706 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:30.226557    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/61782e91-1520-4b83-ad8d-8dd944881b52-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/61782e91-1520-4b83-ad8d-8dd944881b52-workload-certs\") pod \"61782e91-1520-4b83-ad8d-8dd944881b52\" (UID: \"61782e91-1520-4b83-ad8d-8dd944881b52\") "
Oct 30 22:17:30.226706 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:30.226584    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/61782e91-1520-4b83-ad8d-8dd944881b52-kube-api-access-zlv77\" (UniqueName: \"kubernetes.io/projected/61782e91-1520-4b83-ad8d-8dd944881b52-kube-api-access-zlv77\") pod \"61782e91-1520-4b83-ad8d-8dd944881b52\" (UID: \"61782e91-1520-4b83-ad8d-8dd944881b52\") "
Oct 30 22:17:30.226706 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:30.226662    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/61782e91-1520-4b83-ad8d-8dd944881b52-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/61782e91-1520-4b83-ad8d-8dd944881b52-istio-data\") pod \"61782e91-1520-4b83-ad8d-8dd944881b52\" (UID: \"61782e91-1520-4b83-ad8d-8dd944881b52\") "
Oct 30 22:17:30.226976 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:30.226713    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/61782e91-1520-4b83-ad8d-8dd944881b52-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/61782e91-1520-4b83-ad8d-8dd944881b52-istio-podinfo\") pod \"61782e91-1520-4b83-ad8d-8dd944881b52\" (UID: \"61782e91-1520-4b83-ad8d-8dd944881b52\") "
Oct 30 22:17:30.226976 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:30.226768    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/61782e91-1520-4b83-ad8d-8dd944881b52-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/61782e91-1520-4b83-ad8d-8dd944881b52-credential-socket\") pod \"61782e91-1520-4b83-ad8d-8dd944881b52\" (UID: \"61782e91-1520-4b83-ad8d-8dd944881b52\") "
Oct 30 22:17:30.226976 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:30.226820    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/61782e91-1520-4b83-ad8d-8dd944881b52-workload-socket" pod "61782e91-1520-4b83-ad8d-8dd944881b52" (UID: "61782e91-1520-4b83-ad8d-8dd944881b52"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:17:30.226976 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:30.226962    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/61782e91-1520-4b83-ad8d-8dd944881b52-workload-certs" pod "61782e91-1520-4b83-ad8d-8dd944881b52" (UID: "61782e91-1520-4b83-ad8d-8dd944881b52"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:17:30.227215 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:30.227004    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/61782e91-1520-4b83-ad8d-8dd944881b52-istiod-ca-cert" pod "61782e91-1520-4b83-ad8d-8dd944881b52" (UID: "61782e91-1520-4b83-ad8d-8dd944881b52"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:17:30.227215 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:30.227095    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/61782e91-1520-4b83-ad8d-8dd944881b52-istio-data" pod "61782e91-1520-4b83-ad8d-8dd944881b52" (UID: "61782e91-1520-4b83-ad8d-8dd944881b52"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:17:30.227318 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:30.227267    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/61782e91-1520-4b83-ad8d-8dd944881b52-credential-socket" pod "61782e91-1520-4b83-ad8d-8dd944881b52" (UID: "61782e91-1520-4b83-ad8d-8dd944881b52"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:17:30.227485 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:30.227463    4473 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/61782e91-1520-4b83-ad8d-8dd944881b52-workload-certs\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:17:30.227557 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:30.227492    4473 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/61782e91-1520-4b83-ad8d-8dd944881b52-istio-data\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:17:30.227557 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:30.227513    4473 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/61782e91-1520-4b83-ad8d-8dd944881b52-credential-socket\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:17:30.227695 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:30.227625    4473 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/61782e91-1520-4b83-ad8d-8dd944881b52-istiod-ca-cert\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:17:30.227737 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:30.227698    4473 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/61782e91-1520-4b83-ad8d-8dd944881b52-workload-socket\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:17:30.229114 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:30.229089    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/61782e91-1520-4b83-ad8d-8dd944881b52-istio-podinfo" pod "61782e91-1520-4b83-ad8d-8dd944881b52" (UID: "61782e91-1520-4b83-ad8d-8dd944881b52"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:17:30.229228 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:30.229152    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/61782e91-1520-4b83-ad8d-8dd944881b52-kube-api-access-zlv77" pod "61782e91-1520-4b83-ad8d-8dd944881b52" (UID: "61782e91-1520-4b83-ad8d-8dd944881b52"). InnerVolumeSpecName "kube-api-access-zlv77". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:17:30.229373 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:30.229353    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/61782e91-1520-4b83-ad8d-8dd944881b52-istio-token" pod "61782e91-1520-4b83-ad8d-8dd944881b52" (UID: "61782e91-1520-4b83-ad8d-8dd944881b52"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:17:30.229443 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:30.229353    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/61782e91-1520-4b83-ad8d-8dd944881b52-istio-envoy" pod "61782e91-1520-4b83-ad8d-8dd944881b52" (UID: "61782e91-1520-4b83-ad8d-8dd944881b52"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:17:30.330964 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:30.330901    4473 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/61782e91-1520-4b83-ad8d-8dd944881b52-istio-envoy\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:17:30.330964 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:30.330948    4473 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zlv77\" (UniqueName: \"kubernetes.io/projected/61782e91-1520-4b83-ad8d-8dd944881b52-kube-api-access-zlv77\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:17:30.330964 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:30.330959    4473 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/61782e91-1520-4b83-ad8d-8dd944881b52-istio-podinfo\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:17:30.330964 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:30.330969    4473 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/61782e91-1520-4b83-ad8d-8dd944881b52-istio-token\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:17:30.503365 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:30.503274    4473 generic.go:356] "Generic (PLEG): container finished" podID="61782e91-1520-4b83-ad8d-8dd944881b52" containerID="507749823c9e799137e00bbb0c980cdf0023eb744fa7b5df0da742998c5ddc81" exitCode=0
Oct 30 22:17:30.503365 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:30.503328    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w5-e431fc-openshift-default-6d78d86df7-lhwsl"
Oct 30 22:17:30.503593 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:30.503366    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w5-e431fc-openshift-default-6d78d86df7-lhwsl" event={"ID":"61782e91-1520-4b83-ad8d-8dd944881b52","Type":"ContainerDied","Data":"507749823c9e799137e00bbb0c980cdf0023eb744fa7b5df0da742998c5ddc81"}
Oct 30 22:17:30.503593 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:30.503424    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w5-e431fc-openshift-default-6d78d86df7-lhwsl" event={"ID":"61782e91-1520-4b83-ad8d-8dd944881b52","Type":"ContainerDied","Data":"243063e8918ace27f3ed559bbf20257884c012df1d5767c346d129949d67e2fa"}
Oct 30 22:17:30.503593 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:30.503451    4473 scope.go:122] "RemoveContainer" containerID="507749823c9e799137e00bbb0c980cdf0023eb744fa7b5df0da742998c5ddc81"
Oct 30 22:17:30.513770 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:30.513749    4473 scope.go:122] "RemoveContainer" containerID="414face9ec799d471c949aa7a6ab3c8d0c101d52e8254a13d48e49b791e7069e"
Oct 30 22:17:30.523993 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:30.523971    4473 scope.go:122] "RemoveContainer" containerID="507749823c9e799137e00bbb0c980cdf0023eb744fa7b5df0da742998c5ddc81"
Oct 30 22:17:30.524320 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:17:30.524297    4473 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"507749823c9e799137e00bbb0c980cdf0023eb744fa7b5df0da742998c5ddc81\": container with ID starting with 507749823c9e799137e00bbb0c980cdf0023eb744fa7b5df0da742998c5ddc81 not found: ID does not exist" containerID="507749823c9e799137e00bbb0c980cdf0023eb744fa7b5df0da742998c5ddc81"
Oct 30 22:17:30.524392 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:30.524329    4473 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"507749823c9e799137e00bbb0c980cdf0023eb744fa7b5df0da742998c5ddc81"} err="failed to get container status \"507749823c9e799137e00bbb0c980cdf0023eb744fa7b5df0da742998c5ddc81\": rpc error: code = NotFound desc = could not find container \"507749823c9e799137e00bbb0c980cdf0023eb744fa7b5df0da742998c5ddc81\": container with ID starting with 507749823c9e799137e00bbb0c980cdf0023eb744fa7b5df0da742998c5ddc81 not found: ID does not exist"
Oct 30 22:17:30.524392 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:30.524352    4473 scope.go:122] "RemoveContainer" containerID="414face9ec799d471c949aa7a6ab3c8d0c101d52e8254a13d48e49b791e7069e"
Oct 30 22:17:30.524625 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:17:30.524609    4473 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"414face9ec799d471c949aa7a6ab3c8d0c101d52e8254a13d48e49b791e7069e\": container with ID starting with 414face9ec799d471c949aa7a6ab3c8d0c101d52e8254a13d48e49b791e7069e not found: ID does not exist" containerID="414face9ec799d471c949aa7a6ab3c8d0c101d52e8254a13d48e49b791e7069e"
Oct 30 22:17:30.524682 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:30.524636    4473 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"414face9ec799d471c949aa7a6ab3c8d0c101d52e8254a13d48e49b791e7069e"} err="failed to get container status \"414face9ec799d471c949aa7a6ab3c8d0c101d52e8254a13d48e49b791e7069e\": rpc error: code = NotFound desc = could not find container \"414face9ec799d471c949aa7a6ab3c8d0c101d52e8254a13d48e49b791e7069e\": container with ID starting with 414face9ec799d471c949aa7a6ab3c8d0c101d52e8254a13d48e49b791e7069e not found: ID does not exist"
Oct 30 22:17:30.672765 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:30.672729    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w5-e431fc-openshift-default-6d78d86df7-lhwsl"]
Oct 30 22:17:30.677132 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:30.677102    4473 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w5-e431fc-openshift-default-6d78d86df7-lhwsl"]
Oct 30 22:17:32.357035 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:32.357003    4473 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="61782e91-1520-4b83-ad8d-8dd944881b52" path="/var/lib/kubelet/pods/61782e91-1520-4b83-ad8d-8dd944881b52/volumes"
Oct 30 22:17:32.492518 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:32.492484    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-0a7679-745b8695b5-jlj2t"]
Oct 30 22:17:32.492711 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:32.492686    4473 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-0a7679-745b8695b5-jlj2t" podUID="a9e13de9-d3e4-4200-9ac9-6e2ba583b818" containerName="maas-api" containerID="cri-o://791c82775545365f5bd15ff8879ffad9cf414be059e508fd5ac9adf095db17f9" gracePeriod=30
Oct 30 22:17:32.848763 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:32.848737    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-0a7679-745b8695b5-jlj2t"
Oct 30 22:17:32.969531 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:32.969496    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a9e13de9-d3e4-4200-9ac9-6e2ba583b818-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a9e13de9-d3e4-4200-9ac9-6e2ba583b818-openshift-service-ca\") pod \"a9e13de9-d3e4-4200-9ac9-6e2ba583b818\" (UID: \"a9e13de9-d3e4-4200-9ac9-6e2ba583b818\") "
Oct 30 22:17:32.969711 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:32.969577    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a9e13de9-d3e4-4200-9ac9-6e2ba583b818-kube-api-access-nvcpl\" (UniqueName: \"kubernetes.io/projected/a9e13de9-d3e4-4200-9ac9-6e2ba583b818-kube-api-access-nvcpl\") pod \"a9e13de9-d3e4-4200-9ac9-6e2ba583b818\" (UID: \"a9e13de9-d3e4-4200-9ac9-6e2ba583b818\") "
Oct 30 22:17:32.969711 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:32.969656    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a9e13de9-d3e4-4200-9ac9-6e2ba583b818-metrics-certs\" (UniqueName: \"kubernetes.io/secret/a9e13de9-d3e4-4200-9ac9-6e2ba583b818-metrics-certs\") pod \"a9e13de9-d3e4-4200-9ac9-6e2ba583b818\" (UID: \"a9e13de9-d3e4-4200-9ac9-6e2ba583b818\") "
Oct 30 22:17:32.969795 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:32.969747    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a9e13de9-d3e4-4200-9ac9-6e2ba583b818-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a9e13de9-d3e4-4200-9ac9-6e2ba583b818-maas-api-tls\") pod \"a9e13de9-d3e4-4200-9ac9-6e2ba583b818\" (UID: \"a9e13de9-d3e4-4200-9ac9-6e2ba583b818\") "
Oct 30 22:17:32.969912 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:32.969882    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a9e13de9-d3e4-4200-9ac9-6e2ba583b818-openshift-service-ca" pod "a9e13de9-d3e4-4200-9ac9-6e2ba583b818" (UID: "a9e13de9-d3e4-4200-9ac9-6e2ba583b818"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:17:32.970361 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:32.970298    4473 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a9e13de9-d3e4-4200-9ac9-6e2ba583b818-openshift-service-ca\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:17:32.973976 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:32.972111    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a9e13de9-d3e4-4200-9ac9-6e2ba583b818-maas-api-tls" pod "a9e13de9-d3e4-4200-9ac9-6e2ba583b818" (UID: "a9e13de9-d3e4-4200-9ac9-6e2ba583b818"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:17:32.973976 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:32.972194    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a9e13de9-d3e4-4200-9ac9-6e2ba583b818-kube-api-access-nvcpl" pod "a9e13de9-d3e4-4200-9ac9-6e2ba583b818" (UID: "a9e13de9-d3e4-4200-9ac9-6e2ba583b818"). InnerVolumeSpecName "kube-api-access-nvcpl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:17:32.973976 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:32.972416    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a9e13de9-d3e4-4200-9ac9-6e2ba583b818-metrics-certs" pod "a9e13de9-d3e4-4200-9ac9-6e2ba583b818" (UID: "a9e13de9-d3e4-4200-9ac9-6e2ba583b818"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:17:33.071363 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:33.071325    4473 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nvcpl\" (UniqueName: \"kubernetes.io/projected/a9e13de9-d3e4-4200-9ac9-6e2ba583b818-kube-api-access-nvcpl\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:17:33.071363 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:33.071360    4473 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a9e13de9-d3e4-4200-9ac9-6e2ba583b818-metrics-certs\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:17:33.071584 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:33.071373    4473 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a9e13de9-d3e4-4200-9ac9-6e2ba583b818-maas-api-tls\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:17:33.531162 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:33.531126    4473 generic.go:356] "Generic (PLEG): container finished" podID="a9e13de9-d3e4-4200-9ac9-6e2ba583b818" containerID="791c82775545365f5bd15ff8879ffad9cf414be059e508fd5ac9adf095db17f9" exitCode=0
Oct 30 22:17:33.531583 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:33.531183    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-0a7679-745b8695b5-jlj2t"
Oct 30 22:17:33.531583 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:33.531221    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-0a7679-745b8695b5-jlj2t" event={"ID":"a9e13de9-d3e4-4200-9ac9-6e2ba583b818","Type":"ContainerDied","Data":"791c82775545365f5bd15ff8879ffad9cf414be059e508fd5ac9adf095db17f9"}
Oct 30 22:17:33.531583 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:33.531278    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-0a7679-745b8695b5-jlj2t" event={"ID":"a9e13de9-d3e4-4200-9ac9-6e2ba583b818","Type":"ContainerDied","Data":"56d8381ed595c4cccc07c082999f0bdf6a74851033cec7bee9e392efc8ded0e1"}
Oct 30 22:17:33.531583 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:33.531307    4473 scope.go:122] "RemoveContainer" containerID="791c82775545365f5bd15ff8879ffad9cf414be059e508fd5ac9adf095db17f9"
Oct 30 22:17:33.543345 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:33.543322    4473 scope.go:122] "RemoveContainer" containerID="791c82775545365f5bd15ff8879ffad9cf414be059e508fd5ac9adf095db17f9"
Oct 30 22:17:33.543702 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:17:33.543677    4473 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"791c82775545365f5bd15ff8879ffad9cf414be059e508fd5ac9adf095db17f9\": container with ID starting with 791c82775545365f5bd15ff8879ffad9cf414be059e508fd5ac9adf095db17f9 not found: ID does not exist" containerID="791c82775545365f5bd15ff8879ffad9cf414be059e508fd5ac9adf095db17f9"
Oct 30 22:17:33.543776 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:33.543713    4473 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"791c82775545365f5bd15ff8879ffad9cf414be059e508fd5ac9adf095db17f9"} err="failed to get container status \"791c82775545365f5bd15ff8879ffad9cf414be059e508fd5ac9adf095db17f9\": rpc error: code = NotFound desc = could not find container \"791c82775545365f5bd15ff8879ffad9cf414be059e508fd5ac9adf095db17f9\": container with ID starting with 791c82775545365f5bd15ff8879ffad9cf414be059e508fd5ac9adf095db17f9 not found: ID does not exist"
Oct 30 22:17:33.697485 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:33.697445    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-0a7679-745b8695b5-jlj2t"]
Oct 30 22:17:33.747584 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:33.747547    4473 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-0a7679-745b8695b5-jlj2t"]
Oct 30 22:17:34.357736 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:34.357705    4473 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a9e13de9-d3e4-4200-9ac9-6e2ba583b818" path="/var/lib/kubelet/pods/a9e13de9-d3e4-4200-9ac9-6e2ba583b818/volumes"
Oct 30 22:17:37.328701 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:37.328668    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w5-5dff71-7b4c67c84c-xs9qg"]
Oct 30 22:17:42.236702 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:42.236669    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w6-0a7679-openshift-default-5b6c6cc645-p8j4w"]
Oct 30 22:17:42.237329 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:42.236862    4473 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w6-0a7679-openshift-default-5b6c6cc645-p8j4w" podUID="b7789a8c-e252-4a6e-b026-3b5ce2b39b41" containerName="istio-proxy" containerID="cri-o://bbaffd0baa49a1a171003426b82cc8248e39da93ec95fef1ce4c8d1d79e74e2c" gracePeriod=30
Oct 30 22:17:42.333301 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:42.331991    4473 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w5-5dff71-7b4c67c84c-xs9qg" podUID="8bdfde4a-7366-46fd-97c6-6c529c9ce2a3" containerName="payload-pre-processing" containerID="cri-o://f3f9a29e2d0de8c767849797a860f4090da88b8631a0fdf0348916a47410da24" gracePeriod=25
Oct 30 22:17:42.590251 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:42.590214    4473 generic.go:356] "Generic (PLEG): container finished" podID="8bdfde4a-7366-46fd-97c6-6c529c9ce2a3" containerID="f3f9a29e2d0de8c767849797a860f4090da88b8631a0fdf0348916a47410da24" exitCode=0
Oct 30 22:17:42.590393 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:42.590260    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w5-5dff71-7b4c67c84c-xs9qg" event={"ID":"8bdfde4a-7366-46fd-97c6-6c529c9ce2a3","Type":"ContainerDied","Data":"f3f9a29e2d0de8c767849797a860f4090da88b8631a0fdf0348916a47410da24"}
Oct 30 22:17:42.590393 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:42.590296    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w5-5dff71-7b4c67c84c-xs9qg" event={"ID":"8bdfde4a-7366-46fd-97c6-6c529c9ce2a3","Type":"ContainerDied","Data":"8c0c382309db4c2441dca38a9556f9f7508a2a082325a91a32bb6933fa33071a"}
Oct 30 22:17:42.590393 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:42.590312    4473 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8c0c382309db4c2441dca38a9556f9f7508a2a082325a91a32bb6933fa33071a"
Oct 30 22:17:42.600894 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:42.600869    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w5-5dff71-7b4c67c84c-xs9qg"
Oct 30 22:17:42.708669 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:42.708637    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8bdfde4a-7366-46fd-97c6-6c529c9ce2a3-kube-api-access-hvxlk\" (UniqueName: \"kubernetes.io/projected/8bdfde4a-7366-46fd-97c6-6c529c9ce2a3-kube-api-access-hvxlk\") pod \"8bdfde4a-7366-46fd-97c6-6c529c9ce2a3\" (UID: \"8bdfde4a-7366-46fd-97c6-6c529c9ce2a3\") "
Oct 30 22:17:42.708847 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:42.708757    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8bdfde4a-7366-46fd-97c6-6c529c9ce2a3-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8bdfde4a-7366-46fd-97c6-6c529c9ce2a3-plugins-config-volume\") pod \"8bdfde4a-7366-46fd-97c6-6c529c9ce2a3\" (UID: \"8bdfde4a-7366-46fd-97c6-6c529c9ce2a3\") "
Oct 30 22:17:42.709224 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:42.709201    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8bdfde4a-7366-46fd-97c6-6c529c9ce2a3-plugins-config-volume" pod "8bdfde4a-7366-46fd-97c6-6c529c9ce2a3" (UID: "8bdfde4a-7366-46fd-97c6-6c529c9ce2a3"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:17:42.709333 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:42.709319    4473 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8bdfde4a-7366-46fd-97c6-6c529c9ce2a3-plugins-config-volume\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:17:42.711062 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:42.711042    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8bdfde4a-7366-46fd-97c6-6c529c9ce2a3-kube-api-access-hvxlk" pod "8bdfde4a-7366-46fd-97c6-6c529c9ce2a3" (UID: "8bdfde4a-7366-46fd-97c6-6c529c9ce2a3"). InnerVolumeSpecName "kube-api-access-hvxlk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:17:42.811115 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:42.811033    4473 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hvxlk\" (UniqueName: \"kubernetes.io/projected/8bdfde4a-7366-46fd-97c6-6c529c9ce2a3-kube-api-access-hvxlk\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:17:43.596396 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:43.596359    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w5-5dff71-7b4c67c84c-xs9qg"
Oct 30 22:17:43.762970 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:43.762935    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w5-5dff71-7b4c67c84c-xs9qg"]
Oct 30 22:17:43.813082 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:43.813041    4473 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w5-5dff71-7b4c67c84c-xs9qg"]
Oct 30 22:17:43.914256 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:43.914223    4473 patch_prober.go:29] interesting pod/e2e-shared-b-w6-0a7679-openshift-default-5b6c6cc645-p8j4w container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 22:17:43.914428 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:43.914278    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-w6-0a7679-openshift-default-5b6c6cc645-p8j4w" podUID="b7789a8c-e252-4a6e-b026-3b5ce2b39b41" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 22:17:44.357789 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:44.357708    4473 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8bdfde4a-7366-46fd-97c6-6c529c9ce2a3" path="/var/lib/kubelet/pods/8bdfde4a-7366-46fd-97c6-6c529c9ce2a3/volumes"
Oct 30 22:17:44.802697 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:44.802602    4473 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-242ddaf0-5d8df4c987-7lkww"]
Oct 30 22:17:44.808707 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:44.805254    4473 cpu_manager.go:414] "containerMap: removing container" podUID="117cd302-f5bc-4c57-9900-1b6ddb8c5e25" containerName="istio-proxy"
Oct 30 22:17:44.808707 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:44.805283    4473 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="117cd302-f5bc-4c57-9900-1b6ddb8c5e25" containerName="istio-proxy"
Oct 30 22:17:44.808707 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:44.805314    4473 cpu_manager.go:414] "containerMap: removing container" podUID="b78bb642-f809-4102-b4e5-ac485f1ca50d" containerName="payload-pre-processing"
Oct 30 22:17:44.808707 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:44.805322    4473 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b78bb642-f809-4102-b4e5-ac485f1ca50d" containerName="payload-pre-processing"
Oct 30 22:17:44.808707 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:44.805372    4473 cpu_manager.go:414] "containerMap: removing container" podUID="61782e91-1520-4b83-ad8d-8dd944881b52" containerName="istio-proxy"
Oct 30 22:17:44.808707 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:44.805381    4473 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="61782e91-1520-4b83-ad8d-8dd944881b52" containerName="istio-proxy"
Oct 30 22:17:44.808707 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:44.805414    4473 cpu_manager.go:414] "containerMap: removing container" podUID="8bdfde4a-7366-46fd-97c6-6c529c9ce2a3" containerName="payload-pre-processing"
Oct 30 22:17:44.808707 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:44.805422    4473 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8bdfde4a-7366-46fd-97c6-6c529c9ce2a3" containerName="payload-pre-processing"
Oct 30 22:17:44.808707 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:44.805444    4473 cpu_manager.go:414] "containerMap: removing container" podUID="a9e13de9-d3e4-4200-9ac9-6e2ba583b818" containerName="maas-api"
Oct 30 22:17:44.808707 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:44.805452    4473 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a9e13de9-d3e4-4200-9ac9-6e2ba583b818" containerName="maas-api"
Oct 30 22:17:44.808707 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:44.805496    4473 cpu_manager.go:414] "containerMap: removing container" podUID="c15267dd-1a03-40d6-b6b1-ae0dec8cc68f" containerName="istio-proxy"
Oct 30 22:17:44.808707 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:44.805503    4473 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c15267dd-1a03-40d6-b6b1-ae0dec8cc68f" containerName="istio-proxy"
Oct 30 22:17:44.808707 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:44.805526    4473 cpu_manager.go:414] "containerMap: removing container" podUID="c15267dd-1a03-40d6-b6b1-ae0dec8cc68f" containerName="istio-proxy"
Oct 30 22:17:44.808707 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:44.805534    4473 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c15267dd-1a03-40d6-b6b1-ae0dec8cc68f" containerName="istio-proxy"
Oct 30 22:17:44.808707 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:44.805555    4473 cpu_manager.go:414] "containerMap: removing container" podUID="b9f9cb51-3fe0-43f3-9be3-95388ed892c8" containerName="istio-proxy"
Oct 30 22:17:44.808707 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:44.805562    4473 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b9f9cb51-3fe0-43f3-9be3-95388ed892c8" containerName="istio-proxy"
Oct 30 22:17:44.808707 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:44.805594    4473 cpu_manager.go:414] "containerMap: removing container" podUID="806e3639-73c0-4dc2-87bf-4a4a95444e49" containerName="maas-api"
Oct 30 22:17:44.808707 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:44.805660    4473 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="806e3639-73c0-4dc2-87bf-4a4a95444e49" containerName="maas-api"
Oct 30 22:17:44.808707 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:44.806646    4473 memory_manager.go:365] "RemoveStaleState removing state" podUID="117cd302-f5bc-4c57-9900-1b6ddb8c5e25" containerName="istio-proxy"
Oct 30 22:17:44.808707 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:44.806672    4473 memory_manager.go:365] "RemoveStaleState removing state" podUID="b78bb642-f809-4102-b4e5-ac485f1ca50d" containerName="payload-pre-processing"
Oct 30 22:17:44.808707 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:44.806691    4473 memory_manager.go:365] "RemoveStaleState removing state" podUID="61782e91-1520-4b83-ad8d-8dd944881b52" containerName="istio-proxy"
Oct 30 22:17:44.808707 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:44.806703    4473 memory_manager.go:365] "RemoveStaleState removing state" podUID="8bdfde4a-7366-46fd-97c6-6c529c9ce2a3" containerName="payload-pre-processing"
Oct 30 22:17:44.808707 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:44.806719    4473 memory_manager.go:365] "RemoveStaleState removing state" podUID="a9e13de9-d3e4-4200-9ac9-6e2ba583b818" containerName="maas-api"
Oct 30 22:17:44.808707 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:44.806731    4473 memory_manager.go:365] "RemoveStaleState removing state" podUID="c15267dd-1a03-40d6-b6b1-ae0dec8cc68f" containerName="istio-proxy"
Oct 30 22:17:44.808707 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:44.806747    4473 memory_manager.go:365] "RemoveStaleState removing state" podUID="c15267dd-1a03-40d6-b6b1-ae0dec8cc68f" containerName="istio-proxy"
Oct 30 22:17:44.808707 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:44.806763    4473 memory_manager.go:365] "RemoveStaleState removing state" podUID="b9f9cb51-3fe0-43f3-9be3-95388ed892c8" containerName="istio-proxy"
Oct 30 22:17:44.808707 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:44.806781    4473 memory_manager.go:365] "RemoveStaleState removing state" podUID="806e3639-73c0-4dc2-87bf-4a4a95444e49" containerName="maas-api"
Oct 30 22:17:44.817681 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:44.817656    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-242ddaf0-5d8df4c987-7lkww"
Oct 30 22:17:44.864999 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:44.864966    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-242ddaf0-5d8df4c987-7lkww"]
Oct 30 22:17:44.911533 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:44.911498    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-242ddaf0-serving-cert\""
Oct 30 22:17:44.936505 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:44.936471    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0dcc6ba9-67e6-4ec3-9315-305edff7967e-maas-api-tls\") pod \"maas-api-e2e-mt-242ddaf0-5d8df4c987-7lkww\" (UID: \"0dcc6ba9-67e6-4ec3-9315-305edff7967e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-242ddaf0-5d8df4c987-7lkww"
Oct 30 22:17:44.936505 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:44.936513    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f66vg\" (UniqueName: \"kubernetes.io/projected/0dcc6ba9-67e6-4ec3-9315-305edff7967e-kube-api-access-f66vg\") pod \"maas-api-e2e-mt-242ddaf0-5d8df4c987-7lkww\" (UID: \"0dcc6ba9-67e6-4ec3-9315-305edff7967e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-242ddaf0-5d8df4c987-7lkww"
Oct 30 22:17:44.936774 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:44.936589    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0dcc6ba9-67e6-4ec3-9315-305edff7967e-openshift-service-ca\") pod \"maas-api-e2e-mt-242ddaf0-5d8df4c987-7lkww\" (UID: \"0dcc6ba9-67e6-4ec3-9315-305edff7967e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-242ddaf0-5d8df4c987-7lkww"
Oct 30 22:17:44.936774 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:44.936755    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0dcc6ba9-67e6-4ec3-9315-305edff7967e-metrics-certs\") pod \"maas-api-e2e-mt-242ddaf0-5d8df4c987-7lkww\" (UID: \"0dcc6ba9-67e6-4ec3-9315-305edff7967e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-242ddaf0-5d8df4c987-7lkww"
Oct 30 22:17:45.037759 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:45.037720    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0dcc6ba9-67e6-4ec3-9315-305edff7967e-maas-api-tls\") pod \"maas-api-e2e-mt-242ddaf0-5d8df4c987-7lkww\" (UID: \"0dcc6ba9-67e6-4ec3-9315-305edff7967e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-242ddaf0-5d8df4c987-7lkww"
Oct 30 22:17:45.037759 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:45.037762    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f66vg\" (UniqueName: \"kubernetes.io/projected/0dcc6ba9-67e6-4ec3-9315-305edff7967e-kube-api-access-f66vg\") pod \"maas-api-e2e-mt-242ddaf0-5d8df4c987-7lkww\" (UID: \"0dcc6ba9-67e6-4ec3-9315-305edff7967e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-242ddaf0-5d8df4c987-7lkww"
Oct 30 22:17:45.038033 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:45.037800    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0dcc6ba9-67e6-4ec3-9315-305edff7967e-openshift-service-ca\") pod \"maas-api-e2e-mt-242ddaf0-5d8df4c987-7lkww\" (UID: \"0dcc6ba9-67e6-4ec3-9315-305edff7967e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-242ddaf0-5d8df4c987-7lkww"
Oct 30 22:17:45.038033 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:17:45.037885    4473 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-242ddaf0-serving-cert: secret "maas-api-e2e-mt-242ddaf0-serving-cert" not found
Oct 30 22:17:45.038033 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:45.037969    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0dcc6ba9-67e6-4ec3-9315-305edff7967e-metrics-certs\") pod \"maas-api-e2e-mt-242ddaf0-5d8df4c987-7lkww\" (UID: \"0dcc6ba9-67e6-4ec3-9315-305edff7967e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-242ddaf0-5d8df4c987-7lkww"
Oct 30 22:17:45.038172 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:17:45.038054    4473 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0dcc6ba9-67e6-4ec3-9315-305edff7967e-maas-api-tls podName:0dcc6ba9-67e6-4ec3-9315-305edff7967e nodeName:}" failed. No retries permitted until 2026-10-30 22:17:45.538031425 +0000 UTC m=+2111.311399905 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/0dcc6ba9-67e6-4ec3-9315-305edff7967e-maas-api-tls") pod "maas-api-e2e-mt-242ddaf0-5d8df4c987-7lkww" (UID: "0dcc6ba9-67e6-4ec3-9315-305edff7967e") : secret "maas-api-e2e-mt-242ddaf0-serving-cert" not found
Oct 30 22:17:45.038564 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:45.038541    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0dcc6ba9-67e6-4ec3-9315-305edff7967e-openshift-service-ca\") pod \"maas-api-e2e-mt-242ddaf0-5d8df4c987-7lkww\" (UID: \"0dcc6ba9-67e6-4ec3-9315-305edff7967e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-242ddaf0-5d8df4c987-7lkww"
Oct 30 22:17:45.040702 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:45.040678    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0dcc6ba9-67e6-4ec3-9315-305edff7967e-metrics-certs\") pod \"maas-api-e2e-mt-242ddaf0-5d8df4c987-7lkww\" (UID: \"0dcc6ba9-67e6-4ec3-9315-305edff7967e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-242ddaf0-5d8df4c987-7lkww"
Oct 30 22:17:45.109507 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:45.109431    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f66vg\" (UniqueName: \"kubernetes.io/projected/0dcc6ba9-67e6-4ec3-9315-305edff7967e-kube-api-access-f66vg\") pod \"maas-api-e2e-mt-242ddaf0-5d8df4c987-7lkww\" (UID: \"0dcc6ba9-67e6-4ec3-9315-305edff7967e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-242ddaf0-5d8df4c987-7lkww"
Oct 30 22:17:45.544297 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:45.544262    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0dcc6ba9-67e6-4ec3-9315-305edff7967e-maas-api-tls\") pod \"maas-api-e2e-mt-242ddaf0-5d8df4c987-7lkww\" (UID: \"0dcc6ba9-67e6-4ec3-9315-305edff7967e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-242ddaf0-5d8df4c987-7lkww"
Oct 30 22:17:45.546866 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:45.546836    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0dcc6ba9-67e6-4ec3-9315-305edff7967e-maas-api-tls\") pod \"maas-api-e2e-mt-242ddaf0-5d8df4c987-7lkww\" (UID: \"0dcc6ba9-67e6-4ec3-9315-305edff7967e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-242ddaf0-5d8df4c987-7lkww"
Oct 30 22:17:45.733746 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:45.733710    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-242ddaf0-5d8df4c987-7lkww"
Oct 30 22:17:46.122312 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:46.122277    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-242ddaf0-5d8df4c987-7lkww"]
Oct 30 22:17:46.123699 ip-10-0-140-18 kubenswrapper[4473]: W1030 22:17:46.123667    4473 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0dcc6ba9_67e6_4ec3_9315_305edff7967e.slice/crio-1c7d467b145820a421b869e9ed19a20e67defba51688a1456b2afa87685ca3f0 WatchSource:0}: Error finding container 1c7d467b145820a421b869e9ed19a20e67defba51688a1456b2afa87685ca3f0: Status 404 returned error can't find the container with id 1c7d467b145820a421b869e9ed19a20e67defba51688a1456b2afa87685ca3f0
Oct 30 22:17:46.125146 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:46.125126    4473 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 22:17:46.616968 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:46.616893    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-242ddaf0-5d8df4c987-7lkww" event={"ID":"0dcc6ba9-67e6-4ec3-9315-305edff7967e","Type":"ContainerStarted","Data":"1c7d467b145820a421b869e9ed19a20e67defba51688a1456b2afa87685ca3f0"}
Oct 30 22:17:47.507647 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:47.507574    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w6-0a7679-openshift-default-5b6c6cc645-p8j4w"
Oct 30 22:17:47.571786 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:47.571736    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b7789a8c-e252-4a6e-b026-3b5ce2b39b41-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7789a8c-e252-4a6e-b026-3b5ce2b39b41-credential-socket\") pod \"b7789a8c-e252-4a6e-b026-3b5ce2b39b41\" (UID: \"b7789a8c-e252-4a6e-b026-3b5ce2b39b41\") "
Oct 30 22:17:47.572036 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:47.571794    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/b7789a8c-e252-4a6e-b026-3b5ce2b39b41-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b7789a8c-e252-4a6e-b026-3b5ce2b39b41-istio-podinfo\") pod \"b7789a8c-e252-4a6e-b026-3b5ce2b39b41\" (UID: \"b7789a8c-e252-4a6e-b026-3b5ce2b39b41\") "
Oct 30 22:17:47.572036 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:47.571865    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b7789a8c-e252-4a6e-b026-3b5ce2b39b41-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7789a8c-e252-4a6e-b026-3b5ce2b39b41-workload-socket\") pod \"b7789a8c-e252-4a6e-b026-3b5ce2b39b41\" (UID: \"b7789a8c-e252-4a6e-b026-3b5ce2b39b41\") "
Oct 30 22:17:47.572036 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:47.571958    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b7789a8c-e252-4a6e-b026-3b5ce2b39b41-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b7789a8c-e252-4a6e-b026-3b5ce2b39b41-workload-certs\") pod \"b7789a8c-e252-4a6e-b026-3b5ce2b39b41\" (UID: \"b7789a8c-e252-4a6e-b026-3b5ce2b39b41\") "
Oct 30 22:17:47.572220 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:47.572061    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b7789a8c-e252-4a6e-b026-3b5ce2b39b41-kube-api-access-26n5z\" (UniqueName: \"kubernetes.io/projected/b7789a8c-e252-4a6e-b026-3b5ce2b39b41-kube-api-access-26n5z\") pod \"b7789a8c-e252-4a6e-b026-3b5ce2b39b41\" (UID: \"b7789a8c-e252-4a6e-b026-3b5ce2b39b41\") "
Oct 30 22:17:47.572220 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:47.572114    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b7789a8c-e252-4a6e-b026-3b5ce2b39b41-credential-socket" pod "b7789a8c-e252-4a6e-b026-3b5ce2b39b41" (UID: "b7789a8c-e252-4a6e-b026-3b5ce2b39b41"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:17:47.572220 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:47.572134    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b7789a8c-e252-4a6e-b026-3b5ce2b39b41-workload-socket" pod "b7789a8c-e252-4a6e-b026-3b5ce2b39b41" (UID: "b7789a8c-e252-4a6e-b026-3b5ce2b39b41"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:17:47.572220 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:47.572122    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b7789a8c-e252-4a6e-b026-3b5ce2b39b41-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b7789a8c-e252-4a6e-b026-3b5ce2b39b41-istio-envoy\") pod \"b7789a8c-e252-4a6e-b026-3b5ce2b39b41\" (UID: \"b7789a8c-e252-4a6e-b026-3b5ce2b39b41\") "
Oct 30 22:17:47.572220 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:47.572205    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b7789a8c-e252-4a6e-b026-3b5ce2b39b41-istio-token\" (UniqueName: \"kubernetes.io/projected/b7789a8c-e252-4a6e-b026-3b5ce2b39b41-istio-token\") pod \"b7789a8c-e252-4a6e-b026-3b5ce2b39b41\" (UID: \"b7789a8c-e252-4a6e-b026-3b5ce2b39b41\") "
Oct 30 22:17:47.572477 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:47.572257    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b7789a8c-e252-4a6e-b026-3b5ce2b39b41-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b7789a8c-e252-4a6e-b026-3b5ce2b39b41-istio-data\") pod \"b7789a8c-e252-4a6e-b026-3b5ce2b39b41\" (UID: \"b7789a8c-e252-4a6e-b026-3b5ce2b39b41\") "
Oct 30 22:17:47.572477 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:47.572371    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b7789a8c-e252-4a6e-b026-3b5ce2b39b41-workload-certs" pod "b7789a8c-e252-4a6e-b026-3b5ce2b39b41" (UID: "b7789a8c-e252-4a6e-b026-3b5ce2b39b41"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:17:47.572477 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:47.572380    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b7789a8c-e252-4a6e-b026-3b5ce2b39b41-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b7789a8c-e252-4a6e-b026-3b5ce2b39b41-istiod-ca-cert\") pod \"b7789a8c-e252-4a6e-b026-3b5ce2b39b41\" (UID: \"b7789a8c-e252-4a6e-b026-3b5ce2b39b41\") "
Oct 30 22:17:47.572642 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:47.572620    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b7789a8c-e252-4a6e-b026-3b5ce2b39b41-istio-data" pod "b7789a8c-e252-4a6e-b026-3b5ce2b39b41" (UID: "b7789a8c-e252-4a6e-b026-3b5ce2b39b41"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:17:47.572699 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:47.572669    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b7789a8c-e252-4a6e-b026-3b5ce2b39b41-istiod-ca-cert" pod "b7789a8c-e252-4a6e-b026-3b5ce2b39b41" (UID: "b7789a8c-e252-4a6e-b026-3b5ce2b39b41"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:17:47.573009 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:47.572988    4473 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b7789a8c-e252-4a6e-b026-3b5ce2b39b41-istio-data\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:17:47.573119 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:47.573012    4473 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b7789a8c-e252-4a6e-b026-3b5ce2b39b41-istiod-ca-cert\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:17:47.573119 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:47.573030    4473 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7789a8c-e252-4a6e-b026-3b5ce2b39b41-credential-socket\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:17:47.573119 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:47.573043    4473 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7789a8c-e252-4a6e-b026-3b5ce2b39b41-workload-socket\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:17:47.573119 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:47.573057    4473 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b7789a8c-e252-4a6e-b026-3b5ce2b39b41-workload-certs\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:17:47.574780 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:47.574750    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b7789a8c-e252-4a6e-b026-3b5ce2b39b41-istio-token" pod "b7789a8c-e252-4a6e-b026-3b5ce2b39b41" (UID: "b7789a8c-e252-4a6e-b026-3b5ce2b39b41"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:17:47.575107 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:47.575006    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b7789a8c-e252-4a6e-b026-3b5ce2b39b41-kube-api-access-26n5z" pod "b7789a8c-e252-4a6e-b026-3b5ce2b39b41" (UID: "b7789a8c-e252-4a6e-b026-3b5ce2b39b41"). InnerVolumeSpecName "kube-api-access-26n5z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:17:47.575107 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:47.575066    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b7789a8c-e252-4a6e-b026-3b5ce2b39b41-istio-envoy" pod "b7789a8c-e252-4a6e-b026-3b5ce2b39b41" (UID: "b7789a8c-e252-4a6e-b026-3b5ce2b39b41"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:17:47.575107 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:47.575082    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b7789a8c-e252-4a6e-b026-3b5ce2b39b41-istio-podinfo" pod "b7789a8c-e252-4a6e-b026-3b5ce2b39b41" (UID: "b7789a8c-e252-4a6e-b026-3b5ce2b39b41"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:17:47.624961 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:47.624907    4473 generic.go:356] "Generic (PLEG): container finished" podID="b7789a8c-e252-4a6e-b026-3b5ce2b39b41" containerID="bbaffd0baa49a1a171003426b82cc8248e39da93ec95fef1ce4c8d1d79e74e2c" exitCode=0
Oct 30 22:17:47.625160 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:47.624983    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w6-0a7679-openshift-default-5b6c6cc645-p8j4w"
Oct 30 22:17:47.625160 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:47.624970    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w6-0a7679-openshift-default-5b6c6cc645-p8j4w" event={"ID":"b7789a8c-e252-4a6e-b026-3b5ce2b39b41","Type":"ContainerDied","Data":"bbaffd0baa49a1a171003426b82cc8248e39da93ec95fef1ce4c8d1d79e74e2c"}
Oct 30 22:17:47.625160 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:47.625051    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w6-0a7679-openshift-default-5b6c6cc645-p8j4w" event={"ID":"b7789a8c-e252-4a6e-b026-3b5ce2b39b41","Type":"ContainerDied","Data":"9ecde4e0a5e17ad41702e166b8f327db178db8054b3b9c74d12027ef3461123d"}
Oct 30 22:17:47.625160 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:47.625080    4473 scope.go:122] "RemoveContainer" containerID="bbaffd0baa49a1a171003426b82cc8248e39da93ec95fef1ce4c8d1d79e74e2c"
Oct 30 22:17:47.627044 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:47.627011    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-242ddaf0-5d8df4c987-7lkww" event={"ID":"0dcc6ba9-67e6-4ec3-9315-305edff7967e","Type":"ContainerStarted","Data":"5534f02179bc9652216b3fda8156c7573f5bc0d6df7ade7b669d41a834ae41e3"}
Oct 30 22:17:47.627180 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:47.627085    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-242ddaf0-5d8df4c987-7lkww"
Oct 30 22:17:47.636990 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:47.636960    4473 scope.go:122] "RemoveContainer" containerID="29565cd3fb267390ae283bf15cf0534a08cb0883922fd44fc5f909778f42b88b"
Oct 30 22:17:47.647429 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:47.647401    4473 scope.go:122] "RemoveContainer" containerID="bbaffd0baa49a1a171003426b82cc8248e39da93ec95fef1ce4c8d1d79e74e2c"
Oct 30 22:17:47.647752 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:17:47.647731    4473 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bbaffd0baa49a1a171003426b82cc8248e39da93ec95fef1ce4c8d1d79e74e2c\": container with ID starting with bbaffd0baa49a1a171003426b82cc8248e39da93ec95fef1ce4c8d1d79e74e2c not found: ID does not exist" containerID="bbaffd0baa49a1a171003426b82cc8248e39da93ec95fef1ce4c8d1d79e74e2c"
Oct 30 22:17:47.647832 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:47.647764    4473 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bbaffd0baa49a1a171003426b82cc8248e39da93ec95fef1ce4c8d1d79e74e2c"} err="failed to get container status \"bbaffd0baa49a1a171003426b82cc8248e39da93ec95fef1ce4c8d1d79e74e2c\": rpc error: code = NotFound desc = could not find container \"bbaffd0baa49a1a171003426b82cc8248e39da93ec95fef1ce4c8d1d79e74e2c\": container with ID starting with bbaffd0baa49a1a171003426b82cc8248e39da93ec95fef1ce4c8d1d79e74e2c not found: ID does not exist"
Oct 30 22:17:47.647832 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:47.647781    4473 scope.go:122] "RemoveContainer" containerID="29565cd3fb267390ae283bf15cf0534a08cb0883922fd44fc5f909778f42b88b"
Oct 30 22:17:47.648122 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:17:47.648107    4473 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"29565cd3fb267390ae283bf15cf0534a08cb0883922fd44fc5f909778f42b88b\": container with ID starting with 29565cd3fb267390ae283bf15cf0534a08cb0883922fd44fc5f909778f42b88b not found: ID does not exist" containerID="29565cd3fb267390ae283bf15cf0534a08cb0883922fd44fc5f909778f42b88b"
Oct 30 22:17:47.648168 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:47.648126    4473 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"29565cd3fb267390ae283bf15cf0534a08cb0883922fd44fc5f909778f42b88b"} err="failed to get container status \"29565cd3fb267390ae283bf15cf0534a08cb0883922fd44fc5f909778f42b88b\": rpc error: code = NotFound desc = could not find container \"29565cd3fb267390ae283bf15cf0534a08cb0883922fd44fc5f909778f42b88b\": container with ID starting with 29565cd3fb267390ae283bf15cf0534a08cb0883922fd44fc5f909778f42b88b not found: ID does not exist"
Oct 30 22:17:47.674119 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:47.674075    4473 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b7789a8c-e252-4a6e-b026-3b5ce2b39b41-istio-podinfo\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:17:47.674119 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:47.674114    4473 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-26n5z\" (UniqueName: \"kubernetes.io/projected/b7789a8c-e252-4a6e-b026-3b5ce2b39b41-kube-api-access-26n5z\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:17:47.674391 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:47.674131    4473 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b7789a8c-e252-4a6e-b026-3b5ce2b39b41-istio-envoy\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:17:47.674391 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:47.674144    4473 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b7789a8c-e252-4a6e-b026-3b5ce2b39b41-istio-token\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:17:47.796910 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:47.796794    4473 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-242ddaf0-5d8df4c987-7lkww" podStartSLOduration=2.998854998 podStartE2EDuration="3.796773305s" totalImagesPullingTime="797.918307ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:17:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:17:47.793876298 +0000 UTC m=+2113.567244788" watchObservedRunningTime="2026-10-30 22:17:47.796773305 +0000 UTC m=+2113.570141792"
Oct 30 22:17:47.941038 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:47.940997    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w6-0a7679-openshift-default-5b6c6cc645-p8j4w"]
Oct 30 22:17:47.952035 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:47.946484    4473 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w6-0a7679-openshift-default-5b6c6cc645-p8j4w"]
Oct 30 22:17:48.360948 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:48.360256    4473 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b7789a8c-e252-4a6e-b026-3b5ce2b39b41" path="/var/lib/kubelet/pods/b7789a8c-e252-4a6e-b026-3b5ce2b39b41/volumes"
Oct 30 22:17:50.326439 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:17:50.326402    4473 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fncvg" containerName="extractor"
Oct 30 22:17:53.640417 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:53.640387    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-242ddaf0-5d8df4c987-7lkww"
Oct 30 22:17:55.669105 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:55.669063    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-db492e-676764d957-6j782"]
Oct 30 22:17:55.669573 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:55.669296    4473 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-db492e-676764d957-6j782" podUID="8fa4be84-31c4-4076-b5cd-d72fcbbe0e5d" containerName="maas-api" containerID="cri-o://bab83643a56b51bb9fcc7dd530793c7841bf7669d1e5d2ae189fee4fbc5e99d6" gracePeriod=30
Oct 30 22:17:55.980818 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:55.980786    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-db492e-676764d957-6j782"
Oct 30 22:17:56.122451 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:56.122412    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8fa4be84-31c4-4076-b5cd-d72fcbbe0e5d-metrics-certs\" (UniqueName: \"kubernetes.io/secret/8fa4be84-31c4-4076-b5cd-d72fcbbe0e5d-metrics-certs\") pod \"8fa4be84-31c4-4076-b5cd-d72fcbbe0e5d\" (UID: \"8fa4be84-31c4-4076-b5cd-d72fcbbe0e5d\") "
Oct 30 22:17:56.122605 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:56.122580    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8fa4be84-31c4-4076-b5cd-d72fcbbe0e5d-kube-api-access-9hhpk\" (UniqueName: \"kubernetes.io/projected/8fa4be84-31c4-4076-b5cd-d72fcbbe0e5d-kube-api-access-9hhpk\") pod \"8fa4be84-31c4-4076-b5cd-d72fcbbe0e5d\" (UID: \"8fa4be84-31c4-4076-b5cd-d72fcbbe0e5d\") "
Oct 30 22:17:56.122673 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:56.122611    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8fa4be84-31c4-4076-b5cd-d72fcbbe0e5d-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8fa4be84-31c4-4076-b5cd-d72fcbbe0e5d-maas-api-tls\") pod \"8fa4be84-31c4-4076-b5cd-d72fcbbe0e5d\" (UID: \"8fa4be84-31c4-4076-b5cd-d72fcbbe0e5d\") "
Oct 30 22:17:56.122727 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:56.122711    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8fa4be84-31c4-4076-b5cd-d72fcbbe0e5d-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8fa4be84-31c4-4076-b5cd-d72fcbbe0e5d-openshift-service-ca\") pod \"8fa4be84-31c4-4076-b5cd-d72fcbbe0e5d\" (UID: \"8fa4be84-31c4-4076-b5cd-d72fcbbe0e5d\") "
Oct 30 22:17:56.123220 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:56.123190    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8fa4be84-31c4-4076-b5cd-d72fcbbe0e5d-openshift-service-ca" pod "8fa4be84-31c4-4076-b5cd-d72fcbbe0e5d" (UID: "8fa4be84-31c4-4076-b5cd-d72fcbbe0e5d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:17:56.124907 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:56.124869    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8fa4be84-31c4-4076-b5cd-d72fcbbe0e5d-metrics-certs" pod "8fa4be84-31c4-4076-b5cd-d72fcbbe0e5d" (UID: "8fa4be84-31c4-4076-b5cd-d72fcbbe0e5d"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:17:56.125610 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:56.125588    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8fa4be84-31c4-4076-b5cd-d72fcbbe0e5d-kube-api-access-9hhpk" pod "8fa4be84-31c4-4076-b5cd-d72fcbbe0e5d" (UID: "8fa4be84-31c4-4076-b5cd-d72fcbbe0e5d"). InnerVolumeSpecName "kube-api-access-9hhpk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:17:56.126080 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:56.126042    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8fa4be84-31c4-4076-b5cd-d72fcbbe0e5d-maas-api-tls" pod "8fa4be84-31c4-4076-b5cd-d72fcbbe0e5d" (UID: "8fa4be84-31c4-4076-b5cd-d72fcbbe0e5d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:17:56.229190 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:56.229022    4473 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9hhpk\" (UniqueName: \"kubernetes.io/projected/8fa4be84-31c4-4076-b5cd-d72fcbbe0e5d-kube-api-access-9hhpk\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:17:56.229190 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:56.229064    4473 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8fa4be84-31c4-4076-b5cd-d72fcbbe0e5d-maas-api-tls\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:17:56.229190 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:56.229096    4473 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8fa4be84-31c4-4076-b5cd-d72fcbbe0e5d-openshift-service-ca\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:17:56.229190 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:56.229109    4473 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8fa4be84-31c4-4076-b5cd-d72fcbbe0e5d-metrics-certs\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:17:56.704377 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:56.704339    4473 generic.go:356] "Generic (PLEG): container finished" podID="8fa4be84-31c4-4076-b5cd-d72fcbbe0e5d" containerID="bab83643a56b51bb9fcc7dd530793c7841bf7669d1e5d2ae189fee4fbc5e99d6" exitCode=0
Oct 30 22:17:56.704841 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:56.704380    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-db492e-676764d957-6j782" event={"ID":"8fa4be84-31c4-4076-b5cd-d72fcbbe0e5d","Type":"ContainerDied","Data":"bab83643a56b51bb9fcc7dd530793c7841bf7669d1e5d2ae189fee4fbc5e99d6"}
Oct 30 22:17:56.704841 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:56.704439    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-db492e-676764d957-6j782" event={"ID":"8fa4be84-31c4-4076-b5cd-d72fcbbe0e5d","Type":"ContainerDied","Data":"667d07ae516f88876063ebfef0a72156a27b102f33f0350cbe7b4dced2822e04"}
Oct 30 22:17:56.704841 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:56.704470    4473 scope.go:122] "RemoveContainer" containerID="bab83643a56b51bb9fcc7dd530793c7841bf7669d1e5d2ae189fee4fbc5e99d6"
Oct 30 22:17:56.705069 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:56.705046    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-db492e-676764d957-6j782"
Oct 30 22:17:56.715172 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:56.715145    4473 scope.go:122] "RemoveContainer" containerID="bab83643a56b51bb9fcc7dd530793c7841bf7669d1e5d2ae189fee4fbc5e99d6"
Oct 30 22:17:56.715478 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:17:56.715460    4473 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bab83643a56b51bb9fcc7dd530793c7841bf7669d1e5d2ae189fee4fbc5e99d6\": container with ID starting with bab83643a56b51bb9fcc7dd530793c7841bf7669d1e5d2ae189fee4fbc5e99d6 not found: ID does not exist" containerID="bab83643a56b51bb9fcc7dd530793c7841bf7669d1e5d2ae189fee4fbc5e99d6"
Oct 30 22:17:56.715552 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:56.715488    4473 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bab83643a56b51bb9fcc7dd530793c7841bf7669d1e5d2ae189fee4fbc5e99d6"} err="failed to get container status \"bab83643a56b51bb9fcc7dd530793c7841bf7669d1e5d2ae189fee4fbc5e99d6\": rpc error: code = NotFound desc = could not find container \"bab83643a56b51bb9fcc7dd530793c7841bf7669d1e5d2ae189fee4fbc5e99d6\": container with ID starting with bab83643a56b51bb9fcc7dd530793c7841bf7669d1e5d2ae189fee4fbc5e99d6 not found: ID does not exist"
Oct 30 22:17:56.875999 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:56.875963    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-db492e-676764d957-6j782"]
Oct 30 22:17:56.882452 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:56.882421    4473 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-db492e-676764d957-6j782"]
Oct 30 22:17:58.356935 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:17:58.356884    4473 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8fa4be84-31c4-4076-b5cd-d72fcbbe0e5d" path="/var/lib/kubelet/pods/8fa4be84-31c4-4076-b5cd-d72fcbbe0e5d/volumes"
Oct 30 22:18:01.918762 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:01.918718    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-5dff71-5b5d58b5bf-bmk4c"]
Oct 30 22:18:01.919318 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:01.919046    4473 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-5dff71-5b5d58b5bf-bmk4c" podUID="351fd23f-7686-4ce9-903a-95d40b6a521c" containerName="maas-api" containerID="cri-o://5f526a00948d1c9c9af86a3ade99b822b9fb0790a60358610d25d5796a9fd03f" gracePeriod=30
Oct 30 22:18:02.240760 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:02.240727    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-5dff71-5b5d58b5bf-bmk4c"
Oct 30 22:18:02.311663 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:02.311580    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/351fd23f-7686-4ce9-903a-95d40b6a521c-metrics-certs\" (UniqueName: \"kubernetes.io/secret/351fd23f-7686-4ce9-903a-95d40b6a521c-metrics-certs\") pod \"351fd23f-7686-4ce9-903a-95d40b6a521c\" (UID: \"351fd23f-7686-4ce9-903a-95d40b6a521c\") "
Oct 30 22:18:02.311663 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:02.311632    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/351fd23f-7686-4ce9-903a-95d40b6a521c-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/351fd23f-7686-4ce9-903a-95d40b6a521c-openshift-service-ca\") pod \"351fd23f-7686-4ce9-903a-95d40b6a521c\" (UID: \"351fd23f-7686-4ce9-903a-95d40b6a521c\") "
Oct 30 22:18:02.311984 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:02.311727    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/351fd23f-7686-4ce9-903a-95d40b6a521c-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/351fd23f-7686-4ce9-903a-95d40b6a521c-maas-api-tls\") pod \"351fd23f-7686-4ce9-903a-95d40b6a521c\" (UID: \"351fd23f-7686-4ce9-903a-95d40b6a521c\") "
Oct 30 22:18:02.311984 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:02.311754    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/351fd23f-7686-4ce9-903a-95d40b6a521c-kube-api-access-wmpjf\" (UniqueName: \"kubernetes.io/projected/351fd23f-7686-4ce9-903a-95d40b6a521c-kube-api-access-wmpjf\") pod \"351fd23f-7686-4ce9-903a-95d40b6a521c\" (UID: \"351fd23f-7686-4ce9-903a-95d40b6a521c\") "
Oct 30 22:18:02.312149 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:02.312118    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/351fd23f-7686-4ce9-903a-95d40b6a521c-openshift-service-ca" pod "351fd23f-7686-4ce9-903a-95d40b6a521c" (UID: "351fd23f-7686-4ce9-903a-95d40b6a521c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:18:02.312374 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:02.312356    4473 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/351fd23f-7686-4ce9-903a-95d40b6a521c-openshift-service-ca\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:18:02.314163 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:02.314132    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/351fd23f-7686-4ce9-903a-95d40b6a521c-kube-api-access-wmpjf" pod "351fd23f-7686-4ce9-903a-95d40b6a521c" (UID: "351fd23f-7686-4ce9-903a-95d40b6a521c"). InnerVolumeSpecName "kube-api-access-wmpjf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:18:02.314567 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:02.314538    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/351fd23f-7686-4ce9-903a-95d40b6a521c-metrics-certs" pod "351fd23f-7686-4ce9-903a-95d40b6a521c" (UID: "351fd23f-7686-4ce9-903a-95d40b6a521c"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:18:02.314567 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:02.314553    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/351fd23f-7686-4ce9-903a-95d40b6a521c-maas-api-tls" pod "351fd23f-7686-4ce9-903a-95d40b6a521c" (UID: "351fd23f-7686-4ce9-903a-95d40b6a521c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:18:02.414235 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:02.414199    4473 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/351fd23f-7686-4ce9-903a-95d40b6a521c-metrics-certs\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:18:02.414235 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:02.414231    4473 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/351fd23f-7686-4ce9-903a-95d40b6a521c-maas-api-tls\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:18:02.414459 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:02.414243    4473 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wmpjf\" (UniqueName: \"kubernetes.io/projected/351fd23f-7686-4ce9-903a-95d40b6a521c-kube-api-access-wmpjf\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:18:02.768932 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:02.768879    4473 generic.go:356] "Generic (PLEG): container finished" podID="351fd23f-7686-4ce9-903a-95d40b6a521c" containerID="5f526a00948d1c9c9af86a3ade99b822b9fb0790a60358610d25d5796a9fd03f" exitCode=0
Oct 30 22:18:02.769124 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:02.768949    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-5dff71-5b5d58b5bf-bmk4c" event={"ID":"351fd23f-7686-4ce9-903a-95d40b6a521c","Type":"ContainerDied","Data":"5f526a00948d1c9c9af86a3ade99b822b9fb0790a60358610d25d5796a9fd03f"}
Oct 30 22:18:02.769124 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:02.769008    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-5dff71-5b5d58b5bf-bmk4c" event={"ID":"351fd23f-7686-4ce9-903a-95d40b6a521c","Type":"ContainerDied","Data":"c65a1206a24c0f0df1265f048b4ae55c2832dae46784859c9c6c2a16f662175e"}
Oct 30 22:18:02.769124 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:02.769029    4473 scope.go:122] "RemoveContainer" containerID="5f526a00948d1c9c9af86a3ade99b822b9fb0790a60358610d25d5796a9fd03f"
Oct 30 22:18:02.769124 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:02.768960    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-5dff71-5b5d58b5bf-bmk4c"
Oct 30 22:18:02.778839 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:02.778816    4473 scope.go:122] "RemoveContainer" containerID="5f526a00948d1c9c9af86a3ade99b822b9fb0790a60358610d25d5796a9fd03f"
Oct 30 22:18:02.779175 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:18:02.779152    4473 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5f526a00948d1c9c9af86a3ade99b822b9fb0790a60358610d25d5796a9fd03f\": container with ID starting with 5f526a00948d1c9c9af86a3ade99b822b9fb0790a60358610d25d5796a9fd03f not found: ID does not exist" containerID="5f526a00948d1c9c9af86a3ade99b822b9fb0790a60358610d25d5796a9fd03f"
Oct 30 22:18:02.779224 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:02.779186    4473 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5f526a00948d1c9c9af86a3ade99b822b9fb0790a60358610d25d5796a9fd03f"} err="failed to get container status \"5f526a00948d1c9c9af86a3ade99b822b9fb0790a60358610d25d5796a9fd03f\": rpc error: code = NotFound desc = could not find container \"5f526a00948d1c9c9af86a3ade99b822b9fb0790a60358610d25d5796a9fd03f\": container with ID starting with 5f526a00948d1c9c9af86a3ade99b822b9fb0790a60358610d25d5796a9fd03f not found: ID does not exist"
Oct 30 22:18:02.944874 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:02.944837    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-5dff71-5b5d58b5bf-bmk4c"]
Oct 30 22:18:02.960444 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:02.960399    4473 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-5dff71-5b5d58b5bf-bmk4c"]
Oct 30 22:18:04.358414 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:04.358374    4473 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="351fd23f-7686-4ce9-903a-95d40b6a521c" path="/var/lib/kubelet/pods/351fd23f-7686-4ce9-903a-95d40b6a521c/volumes"
Oct 30 22:18:10.830003 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:10.829963    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w5-5dff71-openshift-default-79bd87bcfc-jng5v"]
Oct 30 22:18:10.830740 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:10.830706    4473 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w5-5dff71-openshift-default-79bd87bcfc-jng5v" podUID="eadc7f9e-be79-4abe-9a9d-da311992b192" containerName="istio-proxy" containerID="cri-o://54375898e617206003c59fb2f48160a64a8553ca3d642e860e3cdba8bd78515f" gracePeriod=30
Oct 30 22:18:11.117066 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:11.117032    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-dde1e8/distinct-2-w0-dde1e8-kserve-744857d57-ftt8b"]
Oct 30 22:18:11.126784 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:11.126744    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-dde1e8/distinct-w0-dde1e8-kserve-75b68b4c5d-hts2k"]
Oct 30 22:18:16.109497 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:16.109428    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w5-5dff71-openshift-default-79bd87bcfc-jng5v"
Oct 30 22:18:16.213358 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:16.213324    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/eadc7f9e-be79-4abe-9a9d-da311992b192-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eadc7f9e-be79-4abe-9a9d-da311992b192-istio-podinfo\") pod \"eadc7f9e-be79-4abe-9a9d-da311992b192\" (UID: \"eadc7f9e-be79-4abe-9a9d-da311992b192\") "
Oct 30 22:18:16.213579 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:16.213384    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/eadc7f9e-be79-4abe-9a9d-da311992b192-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eadc7f9e-be79-4abe-9a9d-da311992b192-istio-data\") pod \"eadc7f9e-be79-4abe-9a9d-da311992b192\" (UID: \"eadc7f9e-be79-4abe-9a9d-da311992b192\") "
Oct 30 22:18:16.213579 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:16.213417    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/eadc7f9e-be79-4abe-9a9d-da311992b192-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eadc7f9e-be79-4abe-9a9d-da311992b192-credential-socket\") pod \"eadc7f9e-be79-4abe-9a9d-da311992b192\" (UID: \"eadc7f9e-be79-4abe-9a9d-da311992b192\") "
Oct 30 22:18:16.213579 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:16.213450    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/eadc7f9e-be79-4abe-9a9d-da311992b192-istio-token\" (UniqueName: \"kubernetes.io/projected/eadc7f9e-be79-4abe-9a9d-da311992b192-istio-token\") pod \"eadc7f9e-be79-4abe-9a9d-da311992b192\" (UID: \"eadc7f9e-be79-4abe-9a9d-da311992b192\") "
Oct 30 22:18:16.213579 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:16.213504    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/eadc7f9e-be79-4abe-9a9d-da311992b192-kube-api-access-jnqmk\" (UniqueName: \"kubernetes.io/projected/eadc7f9e-be79-4abe-9a9d-da311992b192-kube-api-access-jnqmk\") pod \"eadc7f9e-be79-4abe-9a9d-da311992b192\" (UID: \"eadc7f9e-be79-4abe-9a9d-da311992b192\") "
Oct 30 22:18:16.213579 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:16.213549    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/eadc7f9e-be79-4abe-9a9d-da311992b192-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eadc7f9e-be79-4abe-9a9d-da311992b192-istio-envoy\") pod \"eadc7f9e-be79-4abe-9a9d-da311992b192\" (UID: \"eadc7f9e-be79-4abe-9a9d-da311992b192\") "
Oct 30 22:18:16.213853 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:16.213592    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/eadc7f9e-be79-4abe-9a9d-da311992b192-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eadc7f9e-be79-4abe-9a9d-da311992b192-istiod-ca-cert\") pod \"eadc7f9e-be79-4abe-9a9d-da311992b192\" (UID: \"eadc7f9e-be79-4abe-9a9d-da311992b192\") "
Oct 30 22:18:16.213853 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:16.213650    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/eadc7f9e-be79-4abe-9a9d-da311992b192-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eadc7f9e-be79-4abe-9a9d-da311992b192-workload-certs\") pod \"eadc7f9e-be79-4abe-9a9d-da311992b192\" (UID: \"eadc7f9e-be79-4abe-9a9d-da311992b192\") "
Oct 30 22:18:16.213853 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:16.213752    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/eadc7f9e-be79-4abe-9a9d-da311992b192-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eadc7f9e-be79-4abe-9a9d-da311992b192-workload-socket\") pod \"eadc7f9e-be79-4abe-9a9d-da311992b192\" (UID: \"eadc7f9e-be79-4abe-9a9d-da311992b192\") "
Oct 30 22:18:16.213853 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:16.213784    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eadc7f9e-be79-4abe-9a9d-da311992b192-credential-socket" pod "eadc7f9e-be79-4abe-9a9d-da311992b192" (UID: "eadc7f9e-be79-4abe-9a9d-da311992b192"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:18:16.213853 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:16.213817    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eadc7f9e-be79-4abe-9a9d-da311992b192-istio-data" pod "eadc7f9e-be79-4abe-9a9d-da311992b192" (UID: "eadc7f9e-be79-4abe-9a9d-da311992b192"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:18:16.214107 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:16.214078    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eadc7f9e-be79-4abe-9a9d-da311992b192-workload-certs" pod "eadc7f9e-be79-4abe-9a9d-da311992b192" (UID: "eadc7f9e-be79-4abe-9a9d-da311992b192"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:18:16.214187 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:16.214162    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eadc7f9e-be79-4abe-9a9d-da311992b192-istiod-ca-cert" pod "eadc7f9e-be79-4abe-9a9d-da311992b192" (UID: "eadc7f9e-be79-4abe-9a9d-da311992b192"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:18:16.214347 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:16.214322    4473 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eadc7f9e-be79-4abe-9a9d-da311992b192-istio-data\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:18:16.214577 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:16.214351    4473 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eadc7f9e-be79-4abe-9a9d-da311992b192-credential-socket\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:18:16.214577 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:16.214366    4473 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eadc7f9e-be79-4abe-9a9d-da311992b192-istiod-ca-cert\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:18:16.214577 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:16.214380    4473 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eadc7f9e-be79-4abe-9a9d-da311992b192-workload-certs\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:18:16.214826 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:16.214672    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eadc7f9e-be79-4abe-9a9d-da311992b192-workload-socket" pod "eadc7f9e-be79-4abe-9a9d-da311992b192" (UID: "eadc7f9e-be79-4abe-9a9d-da311992b192"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:18:16.216039 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:16.216011    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/eadc7f9e-be79-4abe-9a9d-da311992b192-istio-podinfo" pod "eadc7f9e-be79-4abe-9a9d-da311992b192" (UID: "eadc7f9e-be79-4abe-9a9d-da311992b192"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:18:16.216267 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:16.216241    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eadc7f9e-be79-4abe-9a9d-da311992b192-istio-envoy" pod "eadc7f9e-be79-4abe-9a9d-da311992b192" (UID: "eadc7f9e-be79-4abe-9a9d-da311992b192"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:18:16.216475 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:16.216438    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eadc7f9e-be79-4abe-9a9d-da311992b192-kube-api-access-jnqmk" pod "eadc7f9e-be79-4abe-9a9d-da311992b192" (UID: "eadc7f9e-be79-4abe-9a9d-da311992b192"). InnerVolumeSpecName "kube-api-access-jnqmk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:18:16.216589 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:16.216526    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eadc7f9e-be79-4abe-9a9d-da311992b192-istio-token" pod "eadc7f9e-be79-4abe-9a9d-da311992b192" (UID: "eadc7f9e-be79-4abe-9a9d-da311992b192"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:18:16.234775 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:16.234743    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w5-9f8aea/distinct-w5-9f8aea-kserve-5b564579cd-kmxzz"]
Oct 30 22:18:16.316034 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:16.315993    4473 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eadc7f9e-be79-4abe-9a9d-da311992b192-istio-podinfo\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:18:16.316034 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:16.316026    4473 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eadc7f9e-be79-4abe-9a9d-da311992b192-istio-token\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:18:16.316034 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:16.316036    4473 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jnqmk\" (UniqueName: \"kubernetes.io/projected/eadc7f9e-be79-4abe-9a9d-da311992b192-kube-api-access-jnqmk\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:18:16.316281 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:16.316047    4473 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eadc7f9e-be79-4abe-9a9d-da311992b192-istio-envoy\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:18:16.316281 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:16.316055    4473 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eadc7f9e-be79-4abe-9a9d-da311992b192-workload-socket\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:18:16.663552 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:16.663453    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-xct57"]
Oct 30 22:18:16.663754 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:16.663709    4473 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-xct57" podUID="893a5438-3b54-4d77-8675-00260980c0f6" containerName="istio-proxy" containerID="cri-o://1e44b133e1aebad98c1030d44c6e8dc5f3788dba7b582f7f6cd220bcd712a9cd" gracePeriod=30
Oct 30 22:18:16.880424 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:16.880391    4473 generic.go:356] "Generic (PLEG): container finished" podID="eadc7f9e-be79-4abe-9a9d-da311992b192" containerID="54375898e617206003c59fb2f48160a64a8553ca3d642e860e3cdba8bd78515f" exitCode=0
Oct 30 22:18:16.880631 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:16.880441    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w5-5dff71-openshift-default-79bd87bcfc-jng5v"
Oct 30 22:18:16.880631 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:16.880488    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w5-5dff71-openshift-default-79bd87bcfc-jng5v" event={"ID":"eadc7f9e-be79-4abe-9a9d-da311992b192","Type":"ContainerDied","Data":"54375898e617206003c59fb2f48160a64a8553ca3d642e860e3cdba8bd78515f"}
Oct 30 22:18:16.880631 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:16.880559    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w5-5dff71-openshift-default-79bd87bcfc-jng5v" event={"ID":"eadc7f9e-be79-4abe-9a9d-da311992b192","Type":"ContainerDied","Data":"343c2408310ed090c010a9e866158e2191c1999a180a4a6d8d01cddeb33870b4"}
Oct 30 22:18:16.880631 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:16.880589    4473 scope.go:122] "RemoveContainer" containerID="54375898e617206003c59fb2f48160a64a8553ca3d642e860e3cdba8bd78515f"
Oct 30 22:18:16.890511 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:16.890490    4473 scope.go:122] "RemoveContainer" containerID="54375898e617206003c59fb2f48160a64a8553ca3d642e860e3cdba8bd78515f"
Oct 30 22:18:16.890894 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:18:16.890794    4473 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"54375898e617206003c59fb2f48160a64a8553ca3d642e860e3cdba8bd78515f\": container with ID starting with 54375898e617206003c59fb2f48160a64a8553ca3d642e860e3cdba8bd78515f not found: ID does not exist" containerID="54375898e617206003c59fb2f48160a64a8553ca3d642e860e3cdba8bd78515f"
Oct 30 22:18:16.890894 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:16.890833    4473 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"54375898e617206003c59fb2f48160a64a8553ca3d642e860e3cdba8bd78515f"} err="failed to get container status \"54375898e617206003c59fb2f48160a64a8553ca3d642e860e3cdba8bd78515f\": rpc error: code = NotFound desc = could not find container \"54375898e617206003c59fb2f48160a64a8553ca3d642e860e3cdba8bd78515f\": container with ID starting with 54375898e617206003c59fb2f48160a64a8553ca3d642e860e3cdba8bd78515f not found: ID does not exist"
Oct 30 22:18:17.063951 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:17.063822    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w5-5dff71-openshift-default-79bd87bcfc-jng5v"]
Oct 30 22:18:17.076682 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:17.076634    4473 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w5-5dff71-openshift-default-79bd87bcfc-jng5v"]
Oct 30 22:18:18.242393 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:18.242356    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-dde1e8-9575db797-2whh8"]
Oct 30 22:18:18.357199 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:18.357154    4473 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="eadc7f9e-be79-4abe-9a9d-da311992b192" path="/var/lib/kubelet/pods/eadc7f9e-be79-4abe-9a9d-da311992b192/volumes"
Oct 30 22:18:18.357584 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:18.357563    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-dde1e8-59fbfbdb67-vdb8w"]
Oct 30 22:18:20.324589 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:18:20.324552    4473 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fncvg" containerName="extractor"
Oct 30 22:18:21.923222 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:21.923192    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-xct57"
Oct 30 22:18:21.936677 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:21.936595    4473 generic.go:356] "Generic (PLEG): container finished" podID="893a5438-3b54-4d77-8675-00260980c0f6" containerID="1e44b133e1aebad98c1030d44c6e8dc5f3788dba7b582f7f6cd220bcd712a9cd" exitCode=0
Oct 30 22:18:21.936677 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:21.936646    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-xct57"
Oct 30 22:18:21.936866 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:21.936681    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-xct57" event={"ID":"893a5438-3b54-4d77-8675-00260980c0f6","Type":"ContainerDied","Data":"1e44b133e1aebad98c1030d44c6e8dc5f3788dba7b582f7f6cd220bcd712a9cd"}
Oct 30 22:18:21.936866 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:21.936738    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-xct57" event={"ID":"893a5438-3b54-4d77-8675-00260980c0f6","Type":"ContainerDied","Data":"f9e49bacba3d0a169bae3b97fd3bdafefb603f38a3fa86113a62ef474dadf00b"}
Oct 30 22:18:21.936866 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:21.936769    4473 scope.go:122] "RemoveContainer" containerID="1e44b133e1aebad98c1030d44c6e8dc5f3788dba7b582f7f6cd220bcd712a9cd"
Oct 30 22:18:21.947113 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:21.947025    4473 scope.go:122] "RemoveContainer" containerID="21ac28e226e4fd387318bd11bb97c8354829aaf45ba244118fbd6ac71dfb1b36"
Oct 30 22:18:21.962212 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:21.962183    4473 scope.go:122] "RemoveContainer" containerID="1e44b133e1aebad98c1030d44c6e8dc5f3788dba7b582f7f6cd220bcd712a9cd"
Oct 30 22:18:21.962621 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:18:21.962594    4473 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1e44b133e1aebad98c1030d44c6e8dc5f3788dba7b582f7f6cd220bcd712a9cd\": container with ID starting with 1e44b133e1aebad98c1030d44c6e8dc5f3788dba7b582f7f6cd220bcd712a9cd not found: ID does not exist" containerID="1e44b133e1aebad98c1030d44c6e8dc5f3788dba7b582f7f6cd220bcd712a9cd"
Oct 30 22:18:21.962697 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:21.962635    4473 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1e44b133e1aebad98c1030d44c6e8dc5f3788dba7b582f7f6cd220bcd712a9cd"} err="failed to get container status \"1e44b133e1aebad98c1030d44c6e8dc5f3788dba7b582f7f6cd220bcd712a9cd\": rpc error: code = NotFound desc = could not find container \"1e44b133e1aebad98c1030d44c6e8dc5f3788dba7b582f7f6cd220bcd712a9cd\": container with ID starting with 1e44b133e1aebad98c1030d44c6e8dc5f3788dba7b582f7f6cd220bcd712a9cd not found: ID does not exist"
Oct 30 22:18:21.962697 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:21.962663    4473 scope.go:122] "RemoveContainer" containerID="21ac28e226e4fd387318bd11bb97c8354829aaf45ba244118fbd6ac71dfb1b36"
Oct 30 22:18:21.962965 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:18:21.962913    4473 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"21ac28e226e4fd387318bd11bb97c8354829aaf45ba244118fbd6ac71dfb1b36\": container with ID starting with 21ac28e226e4fd387318bd11bb97c8354829aaf45ba244118fbd6ac71dfb1b36 not found: ID does not exist" containerID="21ac28e226e4fd387318bd11bb97c8354829aaf45ba244118fbd6ac71dfb1b36"
Oct 30 22:18:21.963052 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:21.962974    4473 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"21ac28e226e4fd387318bd11bb97c8354829aaf45ba244118fbd6ac71dfb1b36"} err="failed to get container status \"21ac28e226e4fd387318bd11bb97c8354829aaf45ba244118fbd6ac71dfb1b36\": rpc error: code = NotFound desc = could not find container \"21ac28e226e4fd387318bd11bb97c8354829aaf45ba244118fbd6ac71dfb1b36\": container with ID starting with 21ac28e226e4fd387318bd11bb97c8354829aaf45ba244118fbd6ac71dfb1b36 not found: ID does not exist"
Oct 30 22:18:22.084254 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:22.084223    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/893a5438-3b54-4d77-8675-00260980c0f6-istio-token\" (UniqueName: \"kubernetes.io/projected/893a5438-3b54-4d77-8675-00260980c0f6-istio-token\") pod \"893a5438-3b54-4d77-8675-00260980c0f6\" (UID: \"893a5438-3b54-4d77-8675-00260980c0f6\") "
Oct 30 22:18:22.084420 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:22.084317    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/893a5438-3b54-4d77-8675-00260980c0f6-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/893a5438-3b54-4d77-8675-00260980c0f6-credential-socket\") pod \"893a5438-3b54-4d77-8675-00260980c0f6\" (UID: \"893a5438-3b54-4d77-8675-00260980c0f6\") "
Oct 30 22:18:22.084420 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:22.084357    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/893a5438-3b54-4d77-8675-00260980c0f6-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/893a5438-3b54-4d77-8675-00260980c0f6-istio-envoy\") pod \"893a5438-3b54-4d77-8675-00260980c0f6\" (UID: \"893a5438-3b54-4d77-8675-00260980c0f6\") "
Oct 30 22:18:22.084420 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:22.084375    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/893a5438-3b54-4d77-8675-00260980c0f6-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/893a5438-3b54-4d77-8675-00260980c0f6-istio-podinfo\") pod \"893a5438-3b54-4d77-8675-00260980c0f6\" (UID: \"893a5438-3b54-4d77-8675-00260980c0f6\") "
Oct 30 22:18:22.084610 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:22.084427    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/893a5438-3b54-4d77-8675-00260980c0f6-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/893a5438-3b54-4d77-8675-00260980c0f6-workload-socket\") pod \"893a5438-3b54-4d77-8675-00260980c0f6\" (UID: \"893a5438-3b54-4d77-8675-00260980c0f6\") "
Oct 30 22:18:22.084610 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:22.084463    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/893a5438-3b54-4d77-8675-00260980c0f6-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/893a5438-3b54-4d77-8675-00260980c0f6-istio-data\") pod \"893a5438-3b54-4d77-8675-00260980c0f6\" (UID: \"893a5438-3b54-4d77-8675-00260980c0f6\") "
Oct 30 22:18:22.084610 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:22.084500    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/893a5438-3b54-4d77-8675-00260980c0f6-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/893a5438-3b54-4d77-8675-00260980c0f6-istiod-ca-cert\") pod \"893a5438-3b54-4d77-8675-00260980c0f6\" (UID: \"893a5438-3b54-4d77-8675-00260980c0f6\") "
Oct 30 22:18:22.084610 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:22.084528    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/893a5438-3b54-4d77-8675-00260980c0f6-kube-api-access-w8pmk\" (UniqueName: \"kubernetes.io/projected/893a5438-3b54-4d77-8675-00260980c0f6-kube-api-access-w8pmk\") pod \"893a5438-3b54-4d77-8675-00260980c0f6\" (UID: \"893a5438-3b54-4d77-8675-00260980c0f6\") "
Oct 30 22:18:22.084610 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:22.084599    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/893a5438-3b54-4d77-8675-00260980c0f6-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/893a5438-3b54-4d77-8675-00260980c0f6-workload-certs\") pod \"893a5438-3b54-4d77-8675-00260980c0f6\" (UID: \"893a5438-3b54-4d77-8675-00260980c0f6\") "
Oct 30 22:18:22.084888 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:22.084642    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/893a5438-3b54-4d77-8675-00260980c0f6-credential-socket" pod "893a5438-3b54-4d77-8675-00260980c0f6" (UID: "893a5438-3b54-4d77-8675-00260980c0f6"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:18:22.084888 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:22.084755    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/893a5438-3b54-4d77-8675-00260980c0f6-workload-socket" pod "893a5438-3b54-4d77-8675-00260980c0f6" (UID: "893a5438-3b54-4d77-8675-00260980c0f6"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:18:22.085019 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:22.084995    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/893a5438-3b54-4d77-8675-00260980c0f6-istiod-ca-cert" pod "893a5438-3b54-4d77-8675-00260980c0f6" (UID: "893a5438-3b54-4d77-8675-00260980c0f6"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:18:22.085518 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:22.085245    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/893a5438-3b54-4d77-8675-00260980c0f6-istio-data" pod "893a5438-3b54-4d77-8675-00260980c0f6" (UID: "893a5438-3b54-4d77-8675-00260980c0f6"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:18:22.085518 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:22.085326    4473 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/893a5438-3b54-4d77-8675-00260980c0f6-credential-socket\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:18:22.085518 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:22.085352    4473 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/893a5438-3b54-4d77-8675-00260980c0f6-workload-socket\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:18:22.085518 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:22.085367    4473 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/893a5438-3b54-4d77-8675-00260980c0f6-istiod-ca-cert\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:18:22.085896 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:22.085691    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/893a5438-3b54-4d77-8675-00260980c0f6-workload-certs" pod "893a5438-3b54-4d77-8675-00260980c0f6" (UID: "893a5438-3b54-4d77-8675-00260980c0f6"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:18:22.086942 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:22.086904    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/893a5438-3b54-4d77-8675-00260980c0f6-istio-envoy" pod "893a5438-3b54-4d77-8675-00260980c0f6" (UID: "893a5438-3b54-4d77-8675-00260980c0f6"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:18:22.087200 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:22.087177    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/893a5438-3b54-4d77-8675-00260980c0f6-istio-podinfo" pod "893a5438-3b54-4d77-8675-00260980c0f6" (UID: "893a5438-3b54-4d77-8675-00260980c0f6"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:18:22.087403 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:22.087385    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/893a5438-3b54-4d77-8675-00260980c0f6-istio-token" pod "893a5438-3b54-4d77-8675-00260980c0f6" (UID: "893a5438-3b54-4d77-8675-00260980c0f6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:18:22.087604 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:22.087589    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/893a5438-3b54-4d77-8675-00260980c0f6-kube-api-access-w8pmk" pod "893a5438-3b54-4d77-8675-00260980c0f6" (UID: "893a5438-3b54-4d77-8675-00260980c0f6"). InnerVolumeSpecName "kube-api-access-w8pmk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:18:22.186367 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:22.186324    4473 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/893a5438-3b54-4d77-8675-00260980c0f6-istio-data\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:18:22.186367 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:22.186360    4473 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w8pmk\" (UniqueName: \"kubernetes.io/projected/893a5438-3b54-4d77-8675-00260980c0f6-kube-api-access-w8pmk\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:18:22.186593 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:22.186377    4473 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/893a5438-3b54-4d77-8675-00260980c0f6-workload-certs\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:18:22.186593 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:22.186389    4473 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/893a5438-3b54-4d77-8675-00260980c0f6-istio-token\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:18:22.186593 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:22.186401    4473 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/893a5438-3b54-4d77-8675-00260980c0f6-istio-envoy\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:18:22.186593 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:22.186412    4473 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/893a5438-3b54-4d77-8675-00260980c0f6-istio-podinfo\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:18:22.358542 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:22.358510    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-xct57"]
Oct 30 22:18:22.363432 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:22.363399    4473 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w5-9f8aea-openshift-default-8fc8fc847-xct57"]
Oct 30 22:18:23.243445 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:23.243404    4473 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w0-dde1e8-9575db797-2whh8" podUID="35fcc100-bc0e-4c7a-9e34-aa06b86f288a" containerName="payload-processing" containerID="cri-o://212ee3847d8144b97dc7b6de30a8a3ff313835f090a9da740d069666b6c9fd2b" gracePeriod=25
Oct 30 22:18:23.358538 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:23.358479    4473 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-dde1e8-59fbfbdb67-vdb8w" podUID="40fb21f2-c4df-4bfd-b6e9-4198045570be" containerName="payload-pre-processing" containerID="cri-o://1a7b98080bc79bb81666c15e653c237883b2d9eb22ade815b772790a02231b1a" gracePeriod=25
Oct 30 22:18:23.408094 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:23.408063    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w5-9f8aea-df9ff57d7-k7mlk"]
Oct 30 22:18:24.356442 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:24.356405    4473 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="893a5438-3b54-4d77-8675-00260980c0f6" path="/var/lib/kubelet/pods/893a5438-3b54-4d77-8675-00260980c0f6/volumes"
Oct 30 22:18:26.139190 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:26.139144    4473 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-dde1e8/distinct-2-w0-dde1e8-kserve-744857d57-ftt8b" podUID="577b020d-2a6b-4020-863c-cf19d4f0ca3b" containerName="main" containerID="cri-o://9767ba5bd731c4c231fd2880310ee1358a01dce2e52aad070d6a7aca6f695cb7" gracePeriod=45
Oct 30 22:18:26.148570 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:26.148523    4473 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-dde1e8/distinct-w0-dde1e8-kserve-75b68b4c5d-hts2k" podUID="9d72e7c9-657e-4278-8ed9-53adb75bd530" containerName="main" containerID="cri-o://b7be66dc5d185a6a713cc55f004ac81f8c2fa9bba4e8ab5e179e380b3591c0eb" gracePeriod=45
Oct 30 22:18:26.405864 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:26.405785    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-dde1e8/distinct-2-w0-dde1e8-kserve-744857d57-ftt8b"
Oct 30 22:18:26.444656 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:26.444630    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-dde1e8/distinct-w0-dde1e8-kserve-75b68b4c5d-hts2k"
Oct 30 22:18:26.450880 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:26.450860    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9d72e7c9-657e-4278-8ed9-53adb75bd530-home\" (UniqueName: \"kubernetes.io/empty-dir/9d72e7c9-657e-4278-8ed9-53adb75bd530-home\") pod \"9d72e7c9-657e-4278-8ed9-53adb75bd530\" (UID: \"9d72e7c9-657e-4278-8ed9-53adb75bd530\") "
Oct 30 22:18:26.450990 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:26.450936    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9d72e7c9-657e-4278-8ed9-53adb75bd530-tls-certs\" (UniqueName: \"kubernetes.io/secret/9d72e7c9-657e-4278-8ed9-53adb75bd530-tls-certs\") pod \"9d72e7c9-657e-4278-8ed9-53adb75bd530\" (UID: \"9d72e7c9-657e-4278-8ed9-53adb75bd530\") "
Oct 30 22:18:26.450990 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:26.450966    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/577b020d-2a6b-4020-863c-cf19d4f0ca3b-home\" (UniqueName: \"kubernetes.io/empty-dir/577b020d-2a6b-4020-863c-cf19d4f0ca3b-home\") pod \"577b020d-2a6b-4020-863c-cf19d4f0ca3b\" (UID: \"577b020d-2a6b-4020-863c-cf19d4f0ca3b\") "
Oct 30 22:18:26.451069 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:26.451031    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/577b020d-2a6b-4020-863c-cf19d4f0ca3b-tls-certs\" (UniqueName: \"kubernetes.io/secret/577b020d-2a6b-4020-863c-cf19d4f0ca3b-tls-certs\") pod \"577b020d-2a6b-4020-863c-cf19d4f0ca3b\" (UID: \"577b020d-2a6b-4020-863c-cf19d4f0ca3b\") "
Oct 30 22:18:26.451069 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:26.451048    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/577b020d-2a6b-4020-863c-cf19d4f0ca3b-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/577b020d-2a6b-4020-863c-cf19d4f0ca3b-tmp-dir\") pod \"577b020d-2a6b-4020-863c-cf19d4f0ca3b\" (UID: \"577b020d-2a6b-4020-863c-cf19d4f0ca3b\") "
Oct 30 22:18:26.451149 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:26.451090    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/577b020d-2a6b-4020-863c-cf19d4f0ca3b-dshm\" (UniqueName: \"kubernetes.io/empty-dir/577b020d-2a6b-4020-863c-cf19d4f0ca3b-dshm\") pod \"577b020d-2a6b-4020-863c-cf19d4f0ca3b\" (UID: \"577b020d-2a6b-4020-863c-cf19d4f0ca3b\") "
Oct 30 22:18:26.451149 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:26.451114    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9d72e7c9-657e-4278-8ed9-53adb75bd530-kube-api-access-jvvdr\" (UniqueName: \"kubernetes.io/projected/9d72e7c9-657e-4278-8ed9-53adb75bd530-kube-api-access-jvvdr\") pod \"9d72e7c9-657e-4278-8ed9-53adb75bd530\" (UID: \"9d72e7c9-657e-4278-8ed9-53adb75bd530\") "
Oct 30 22:18:26.451149 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:26.451117    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d72e7c9-657e-4278-8ed9-53adb75bd530-home" pod "9d72e7c9-657e-4278-8ed9-53adb75bd530" (UID: "9d72e7c9-657e-4278-8ed9-53adb75bd530"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:18:26.451149 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:26.451143    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9d72e7c9-657e-4278-8ed9-53adb75bd530-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9d72e7c9-657e-4278-8ed9-53adb75bd530-tmp-dir\") pod \"9d72e7c9-657e-4278-8ed9-53adb75bd530\" (UID: \"9d72e7c9-657e-4278-8ed9-53adb75bd530\") "
Oct 30 22:18:26.451433 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:26.451188    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9d72e7c9-657e-4278-8ed9-53adb75bd530-dshm\" (UniqueName: \"kubernetes.io/empty-dir/9d72e7c9-657e-4278-8ed9-53adb75bd530-dshm\") pod \"9d72e7c9-657e-4278-8ed9-53adb75bd530\" (UID: \"9d72e7c9-657e-4278-8ed9-53adb75bd530\") "
Oct 30 22:18:26.451433 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:26.451262    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9d72e7c9-657e-4278-8ed9-53adb75bd530-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9d72e7c9-657e-4278-8ed9-53adb75bd530-model-cache\") pod \"9d72e7c9-657e-4278-8ed9-53adb75bd530\" (UID: \"9d72e7c9-657e-4278-8ed9-53adb75bd530\") "
Oct 30 22:18:26.451433 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:26.451291    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/577b020d-2a6b-4020-863c-cf19d4f0ca3b-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/577b020d-2a6b-4020-863c-cf19d4f0ca3b-model-cache\") pod \"577b020d-2a6b-4020-863c-cf19d4f0ca3b\" (UID: \"577b020d-2a6b-4020-863c-cf19d4f0ca3b\") "
Oct 30 22:18:26.451433 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:26.451309    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/577b020d-2a6b-4020-863c-cf19d4f0ca3b-home" pod "577b020d-2a6b-4020-863c-cf19d4f0ca3b" (UID: "577b020d-2a6b-4020-863c-cf19d4f0ca3b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:18:26.451433 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:26.451338    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/577b020d-2a6b-4020-863c-cf19d4f0ca3b-kube-api-access-qlhjc\" (UniqueName: \"kubernetes.io/projected/577b020d-2a6b-4020-863c-cf19d4f0ca3b-kube-api-access-qlhjc\") pod \"577b020d-2a6b-4020-863c-cf19d4f0ca3b\" (UID: \"577b020d-2a6b-4020-863c-cf19d4f0ca3b\") "
Oct 30 22:18:26.451433 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:26.451374    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/577b020d-2a6b-4020-863c-cf19d4f0ca3b-tmp-dir" pod "577b020d-2a6b-4020-863c-cf19d4f0ca3b" (UID: "577b020d-2a6b-4020-863c-cf19d4f0ca3b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:18:26.451771 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:26.451538    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d72e7c9-657e-4278-8ed9-53adb75bd530-model-cache" pod "9d72e7c9-657e-4278-8ed9-53adb75bd530" (UID: "9d72e7c9-657e-4278-8ed9-53adb75bd530"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:18:26.451771 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:26.451612    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d72e7c9-657e-4278-8ed9-53adb75bd530-tmp-dir" pod "9d72e7c9-657e-4278-8ed9-53adb75bd530" (UID: "9d72e7c9-657e-4278-8ed9-53adb75bd530"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:18:26.452225 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:26.452151    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/577b020d-2a6b-4020-863c-cf19d4f0ca3b-model-cache" pod "577b020d-2a6b-4020-863c-cf19d4f0ca3b" (UID: "577b020d-2a6b-4020-863c-cf19d4f0ca3b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:18:26.452225 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:26.452189    4473 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/577b020d-2a6b-4020-863c-cf19d4f0ca3b-tmp-dir\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:18:26.452225 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:26.452209    4473 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9d72e7c9-657e-4278-8ed9-53adb75bd530-tmp-dir\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:18:26.452225 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:26.452223    4473 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9d72e7c9-657e-4278-8ed9-53adb75bd530-model-cache\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:18:26.452225 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:26.452235    4473 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9d72e7c9-657e-4278-8ed9-53adb75bd530-home\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:18:26.452225 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:26.452249    4473 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/577b020d-2a6b-4020-863c-cf19d4f0ca3b-home\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:18:26.454232 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:26.454208    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9d72e7c9-657e-4278-8ed9-53adb75bd530-kube-api-access-jvvdr" pod "9d72e7c9-657e-4278-8ed9-53adb75bd530" (UID: "9d72e7c9-657e-4278-8ed9-53adb75bd530"). InnerVolumeSpecName "kube-api-access-jvvdr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:18:26.454327 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:26.454212    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9d72e7c9-657e-4278-8ed9-53adb75bd530-tls-certs" pod "9d72e7c9-657e-4278-8ed9-53adb75bd530" (UID: "9d72e7c9-657e-4278-8ed9-53adb75bd530"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:18:26.454429 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:26.454413    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/577b020d-2a6b-4020-863c-cf19d4f0ca3b-kube-api-access-qlhjc" pod "577b020d-2a6b-4020-863c-cf19d4f0ca3b" (UID: "577b020d-2a6b-4020-863c-cf19d4f0ca3b"). InnerVolumeSpecName "kube-api-access-qlhjc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:18:26.454594 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:26.454579    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d72e7c9-657e-4278-8ed9-53adb75bd530-dshm" pod "9d72e7c9-657e-4278-8ed9-53adb75bd530" (UID: "9d72e7c9-657e-4278-8ed9-53adb75bd530"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:18:26.454733 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:26.454716    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/577b020d-2a6b-4020-863c-cf19d4f0ca3b-dshm" pod "577b020d-2a6b-4020-863c-cf19d4f0ca3b" (UID: "577b020d-2a6b-4020-863c-cf19d4f0ca3b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:18:26.454773 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:26.454726    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/577b020d-2a6b-4020-863c-cf19d4f0ca3b-tls-certs" pod "577b020d-2a6b-4020-863c-cf19d4f0ca3b" (UID: "577b020d-2a6b-4020-863c-cf19d4f0ca3b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:18:26.554183 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:26.554143    4473 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/577b020d-2a6b-4020-863c-cf19d4f0ca3b-dshm\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:18:26.554183 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:26.554184    4473 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jvvdr\" (UniqueName: \"kubernetes.io/projected/9d72e7c9-657e-4278-8ed9-53adb75bd530-kube-api-access-jvvdr\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:18:26.554475 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:26.554199    4473 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9d72e7c9-657e-4278-8ed9-53adb75bd530-dshm\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:18:26.554475 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:26.554212    4473 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/577b020d-2a6b-4020-863c-cf19d4f0ca3b-model-cache\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:18:26.554475 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:26.554225    4473 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qlhjc\" (UniqueName: \"kubernetes.io/projected/577b020d-2a6b-4020-863c-cf19d4f0ca3b-kube-api-access-qlhjc\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:18:26.554475 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:26.554238    4473 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9d72e7c9-657e-4278-8ed9-53adb75bd530-tls-certs\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:18:26.554475 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:26.554250    4473 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/577b020d-2a6b-4020-863c-cf19d4f0ca3b-tls-certs\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:18:26.972284 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:26.972253    4473 generic.go:356] "Generic (PLEG): container finished" podID="9d72e7c9-657e-4278-8ed9-53adb75bd530" containerID="b7be66dc5d185a6a713cc55f004ac81f8c2fa9bba4e8ab5e179e380b3591c0eb" exitCode=0
Oct 30 22:18:26.972485 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:26.972304    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-dde1e8/distinct-w0-dde1e8-kserve-75b68b4c5d-hts2k"
Oct 30 22:18:26.972485 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:26.972338    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-dde1e8/distinct-w0-dde1e8-kserve-75b68b4c5d-hts2k" event={"ID":"9d72e7c9-657e-4278-8ed9-53adb75bd530","Type":"ContainerDied","Data":"b7be66dc5d185a6a713cc55f004ac81f8c2fa9bba4e8ab5e179e380b3591c0eb"}
Oct 30 22:18:26.972485 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:26.972379    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-dde1e8/distinct-w0-dde1e8-kserve-75b68b4c5d-hts2k" event={"ID":"9d72e7c9-657e-4278-8ed9-53adb75bd530","Type":"ContainerDied","Data":"b6d4387db14b77d3c4275d3439b480a5e9ecb1bdd6f2a5a99addb82db8c39cf5"}
Oct 30 22:18:26.972485 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:26.972399    4473 scope.go:122] "RemoveContainer" containerID="b7be66dc5d185a6a713cc55f004ac81f8c2fa9bba4e8ab5e179e380b3591c0eb"
Oct 30 22:18:26.974443 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:26.974418    4473 generic.go:356] "Generic (PLEG): container finished" podID="577b020d-2a6b-4020-863c-cf19d4f0ca3b" containerID="9767ba5bd731c4c231fd2880310ee1358a01dce2e52aad070d6a7aca6f695cb7" exitCode=0
Oct 30 22:18:26.974586 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:26.974472    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-dde1e8/distinct-2-w0-dde1e8-kserve-744857d57-ftt8b"
Oct 30 22:18:26.974586 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:26.974531    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-dde1e8/distinct-2-w0-dde1e8-kserve-744857d57-ftt8b" event={"ID":"577b020d-2a6b-4020-863c-cf19d4f0ca3b","Type":"ContainerDied","Data":"9767ba5bd731c4c231fd2880310ee1358a01dce2e52aad070d6a7aca6f695cb7"}
Oct 30 22:18:26.974586 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:26.974579    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-dde1e8/distinct-2-w0-dde1e8-kserve-744857d57-ftt8b" event={"ID":"577b020d-2a6b-4020-863c-cf19d4f0ca3b","Type":"ContainerDied","Data":"223053fa58ab5cc28c870d9b12d92317c41740a6e04ca701d68f98e0b3fc9d7d"}
Oct 30 22:18:26.982532 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:26.982511    4473 scope.go:122] "RemoveContainer" containerID="b7be66dc5d185a6a713cc55f004ac81f8c2fa9bba4e8ab5e179e380b3591c0eb"
Oct 30 22:18:26.982823 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:18:26.982801    4473 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b7be66dc5d185a6a713cc55f004ac81f8c2fa9bba4e8ab5e179e380b3591c0eb\": container with ID starting with b7be66dc5d185a6a713cc55f004ac81f8c2fa9bba4e8ab5e179e380b3591c0eb not found: ID does not exist" containerID="b7be66dc5d185a6a713cc55f004ac81f8c2fa9bba4e8ab5e179e380b3591c0eb"
Oct 30 22:18:26.982913 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:26.982836    4473 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b7be66dc5d185a6a713cc55f004ac81f8c2fa9bba4e8ab5e179e380b3591c0eb"} err="failed to get container status \"b7be66dc5d185a6a713cc55f004ac81f8c2fa9bba4e8ab5e179e380b3591c0eb\": rpc error: code = NotFound desc = could not find container \"b7be66dc5d185a6a713cc55f004ac81f8c2fa9bba4e8ab5e179e380b3591c0eb\": container with ID starting with b7be66dc5d185a6a713cc55f004ac81f8c2fa9bba4e8ab5e179e380b3591c0eb not found: ID does not exist"
Oct 30 22:18:26.982913 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:26.982859    4473 scope.go:122] "RemoveContainer" containerID="9767ba5bd731c4c231fd2880310ee1358a01dce2e52aad070d6a7aca6f695cb7"
Oct 30 22:18:26.991659 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:26.991641    4473 scope.go:122] "RemoveContainer" containerID="9767ba5bd731c4c231fd2880310ee1358a01dce2e52aad070d6a7aca6f695cb7"
Oct 30 22:18:26.991934 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:18:26.991901    4473 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9767ba5bd731c4c231fd2880310ee1358a01dce2e52aad070d6a7aca6f695cb7\": container with ID starting with 9767ba5bd731c4c231fd2880310ee1358a01dce2e52aad070d6a7aca6f695cb7 not found: ID does not exist" containerID="9767ba5bd731c4c231fd2880310ee1358a01dce2e52aad070d6a7aca6f695cb7"
Oct 30 22:18:26.992016 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:26.991949    4473 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9767ba5bd731c4c231fd2880310ee1358a01dce2e52aad070d6a7aca6f695cb7"} err="failed to get container status \"9767ba5bd731c4c231fd2880310ee1358a01dce2e52aad070d6a7aca6f695cb7\": rpc error: code = NotFound desc = could not find container \"9767ba5bd731c4c231fd2880310ee1358a01dce2e52aad070d6a7aca6f695cb7\": container with ID starting with 9767ba5bd731c4c231fd2880310ee1358a01dce2e52aad070d6a7aca6f695cb7 not found: ID does not exist"
Oct 30 22:18:27.138797 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:27.138753    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-dde1e8/distinct-2-w0-dde1e8-kserve-744857d57-ftt8b"]
Oct 30 22:18:27.142589 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:27.142561    4473 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-dde1e8/distinct-2-w0-dde1e8-kserve-744857d57-ftt8b"]
Oct 30 22:18:27.254536 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:27.254449    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-dde1e8/distinct-w0-dde1e8-kserve-75b68b4c5d-hts2k"]
Oct 30 22:18:27.256622 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:27.256600    4473 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-dde1e8/distinct-w0-dde1e8-kserve-75b68b4c5d-hts2k"]
Oct 30 22:18:28.356161 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:28.356130    4473 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="577b020d-2a6b-4020-863c-cf19d4f0ca3b" path="/var/lib/kubelet/pods/577b020d-2a6b-4020-863c-cf19d4f0ca3b/volumes"
Oct 30 22:18:28.356557 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:28.356542    4473 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9d72e7c9-657e-4278-8ed9-53adb75bd530" path="/var/lib/kubelet/pods/9d72e7c9-657e-4278-8ed9-53adb75bd530/volumes"
Oct 30 22:18:28.409245 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:28.409175    4473 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-9f8aea-df9ff57d7-k7mlk" podUID="7665007f-2e07-47f7-ac42-82dfd53d85a1" containerName="payload-pre-processing" containerID="cri-o://533a86cb750ea2e47f0f08fd67a09ed43ebe877cddb816f28ed25be603352c5c" gracePeriod=25
Oct 30 22:18:28.505513 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:28.505474    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-9f8aea-df9ff57d7-k7mlk" podUID="7665007f-2e07-47f7-ac42-82dfd53d85a1" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:18:29.766662 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:29.766621    4473 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7314-bb536d"]
Oct 30 22:18:29.767372 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:29.767348    4473 cpu_manager.go:414] "containerMap: removing container" podUID="eadc7f9e-be79-4abe-9a9d-da311992b192" containerName="istio-proxy"
Oct 30 22:18:29.767458 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:29.767374    4473 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="eadc7f9e-be79-4abe-9a9d-da311992b192" containerName="istio-proxy"
Oct 30 22:18:29.767458 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:29.767413    4473 cpu_manager.go:414] "containerMap: removing container" podUID="893a5438-3b54-4d77-8675-00260980c0f6" containerName="istio-proxy"
Oct 30 22:18:29.767458 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:29.767421    4473 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="893a5438-3b54-4d77-8675-00260980c0f6" containerName="istio-proxy"
Oct 30 22:18:29.767458 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:29.767431    4473 cpu_manager.go:414] "containerMap: removing container" podUID="8fa4be84-31c4-4076-b5cd-d72fcbbe0e5d" containerName="maas-api"
Oct 30 22:18:29.767458 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:29.767440    4473 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8fa4be84-31c4-4076-b5cd-d72fcbbe0e5d" containerName="maas-api"
Oct 30 22:18:29.767458 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:29.767457    4473 cpu_manager.go:414] "containerMap: removing container" podUID="61782e91-1520-4b83-ad8d-8dd944881b52" containerName="istio-proxy"
Oct 30 22:18:29.767767 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:29.767464    4473 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="61782e91-1520-4b83-ad8d-8dd944881b52" containerName="istio-proxy"
Oct 30 22:18:29.767767 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:29.767480    4473 cpu_manager.go:414] "containerMap: removing container" podUID="b7789a8c-e252-4a6e-b026-3b5ce2b39b41" containerName="istio-proxy"
Oct 30 22:18:29.767767 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:29.767487    4473 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b7789a8c-e252-4a6e-b026-3b5ce2b39b41" containerName="istio-proxy"
Oct 30 22:18:29.767767 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:29.767507    4473 cpu_manager.go:414] "containerMap: removing container" podUID="351fd23f-7686-4ce9-903a-95d40b6a521c" containerName="maas-api"
Oct 30 22:18:29.767767 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:29.767515    4473 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="351fd23f-7686-4ce9-903a-95d40b6a521c" containerName="maas-api"
Oct 30 22:18:29.767767 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:29.767529    4473 cpu_manager.go:414] "containerMap: removing container" podUID="b7789a8c-e252-4a6e-b026-3b5ce2b39b41" containerName="istio-proxy"
Oct 30 22:18:29.767767 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:29.767537    4473 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b7789a8c-e252-4a6e-b026-3b5ce2b39b41" containerName="istio-proxy"
Oct 30 22:18:29.767767 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:29.767550    4473 cpu_manager.go:414] "containerMap: removing container" podUID="9d72e7c9-657e-4278-8ed9-53adb75bd530" containerName="main"
Oct 30 22:18:29.767767 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:29.767558    4473 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9d72e7c9-657e-4278-8ed9-53adb75bd530" containerName="main"
Oct 30 22:18:29.767767 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:29.767569    4473 cpu_manager.go:414] "containerMap: removing container" podUID="577b020d-2a6b-4020-863c-cf19d4f0ca3b" containerName="main"
Oct 30 22:18:29.767767 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:29.767577    4473 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="577b020d-2a6b-4020-863c-cf19d4f0ca3b" containerName="main"
Oct 30 22:18:29.767767 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:29.767595    4473 cpu_manager.go:414] "containerMap: removing container" podUID="893a5438-3b54-4d77-8675-00260980c0f6" containerName="istio-proxy"
Oct 30 22:18:29.767767 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:29.767602    4473 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="893a5438-3b54-4d77-8675-00260980c0f6" containerName="istio-proxy"
Oct 30 22:18:29.767767 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:29.767727    4473 memory_manager.go:365] "RemoveStaleState removing state" podUID="893a5438-3b54-4d77-8675-00260980c0f6" containerName="istio-proxy"
Oct 30 22:18:29.767767 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:29.767740    4473 memory_manager.go:365] "RemoveStaleState removing state" podUID="8fa4be84-31c4-4076-b5cd-d72fcbbe0e5d" containerName="maas-api"
Oct 30 22:18:29.767767 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:29.767752    4473 memory_manager.go:365] "RemoveStaleState removing state" podUID="61782e91-1520-4b83-ad8d-8dd944881b52" containerName="istio-proxy"
Oct 30 22:18:29.767767 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:29.767761    4473 memory_manager.go:365] "RemoveStaleState removing state" podUID="b7789a8c-e252-4a6e-b026-3b5ce2b39b41" containerName="istio-proxy"
Oct 30 22:18:29.767767 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:29.767771    4473 memory_manager.go:365] "RemoveStaleState removing state" podUID="351fd23f-7686-4ce9-903a-95d40b6a521c" containerName="maas-api"
Oct 30 22:18:29.768633 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:29.767780    4473 memory_manager.go:365] "RemoveStaleState removing state" podUID="b7789a8c-e252-4a6e-b026-3b5ce2b39b41" containerName="istio-proxy"
Oct 30 22:18:29.768633 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:29.767789    4473 memory_manager.go:365] "RemoveStaleState removing state" podUID="9d72e7c9-657e-4278-8ed9-53adb75bd530" containerName="main"
Oct 30 22:18:29.768633 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:29.767796    4473 memory_manager.go:365] "RemoveStaleState removing state" podUID="577b020d-2a6b-4020-863c-cf19d4f0ca3b" containerName="main"
Oct 30 22:18:29.768633 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:29.767807    4473 memory_manager.go:365] "RemoveStaleState removing state" podUID="eadc7f9e-be79-4abe-9a9d-da311992b192" containerName="istio-proxy"
Oct 30 22:18:29.771602 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:29.771578    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7314-bb536d"
Oct 30 22:18:29.791512 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:29.791476    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vnvgt\" (UniqueName: \"kubernetes.io/projected/55ac5838-c7d2-4ed9-9ab0-3923413b0392-kube-api-access-vnvgt\") pod \"test-curl-7314-bb536d\" (UID: \"55ac5838-c7d2-4ed9-9ab0-3923413b0392\") " pod="opendatahub/test-curl-7314-bb536d"
Oct 30 22:18:29.824847 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:29.824815    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7314-bb536d"]
Oct 30 22:18:29.831870 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:29.831833    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-qmjfd\""
Oct 30 22:18:29.900166 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:29.900124    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vnvgt\" (UniqueName: \"kubernetes.io/projected/55ac5838-c7d2-4ed9-9ab0-3923413b0392-kube-api-access-vnvgt\") pod \"test-curl-7314-bb536d\" (UID: \"55ac5838-c7d2-4ed9-9ab0-3923413b0392\") " pod="opendatahub/test-curl-7314-bb536d"
Oct 30 22:18:29.959614 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:29.959574    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vnvgt\" (UniqueName: \"kubernetes.io/projected/55ac5838-c7d2-4ed9-9ab0-3923413b0392-kube-api-access-vnvgt\") pod \"test-curl-7314-bb536d\" (UID: \"55ac5838-c7d2-4ed9-9ab0-3923413b0392\") " pod="opendatahub/test-curl-7314-bb536d"
Oct 30 22:18:30.084693 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:30.084605    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7314-bb536d"
Oct 30 22:18:30.463538 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:30.463508    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7314-bb536d"]
Oct 30 22:18:30.464409 ip-10-0-140-18 kubenswrapper[4473]: W1030 22:18:30.464382    4473 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod55ac5838_c7d2_4ed9_9ab0_3923413b0392.slice/crio-fb9fd9ca8c3881633b281751e05267c2bf7f64d3b3aad1678c06484ca6a65f7d WatchSource:0}: Error finding container fb9fd9ca8c3881633b281751e05267c2bf7f64d3b3aad1678c06484ca6a65f7d: Status 404 returned error can't find the container with id fb9fd9ca8c3881633b281751e05267c2bf7f64d3b3aad1678c06484ca6a65f7d
Oct 30 22:18:31.001658 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:31.001620    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7314-bb536d" event={"ID":"55ac5838-c7d2-4ed9-9ab0-3923413b0392","Type":"ContainerStarted","Data":"ff5145ea0052ae2611086c7039ce5bdb4d84287ab95a83deb3eb58e5a0d91ea4"}
Oct 30 22:18:31.002128 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:31.001671    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7314-bb536d" event={"ID":"55ac5838-c7d2-4ed9-9ab0-3923413b0392","Type":"ContainerStarted","Data":"fb9fd9ca8c3881633b281751e05267c2bf7f64d3b3aad1678c06484ca6a65f7d"}
Oct 30 22:18:31.134044 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:31.133978    4473 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/test-curl-7314-bb536d" podStartSLOduration=2.133962546 podStartE2EDuration="2.133962546s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:18:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:18:31.129798333 +0000 UTC m=+2156.903166820" watchObservedRunningTime="2026-10-30 22:18:31.133962546 +0000 UTC m=+2156.907331033"
Oct 30 22:18:31.256588 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:31.256501    4473 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w5-9f8aea/distinct-w5-9f8aea-kserve-5b564579cd-kmxzz" podUID="b11e8b11-334f-4a9e-98c2-b0164567a8af" containerName="main" containerID="cri-o://23e7478f0e398bb331ec0933335a676ae4bb1b251a6f49d13a25042b296a8bcb" gracePeriod=45
Oct 30 22:18:31.318067 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:31.318026    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-9f8aea/distinct-w5-9f8aea-kserve-5b564579cd-kmxzz" podUID="b11e8b11-334f-4a9e-98c2-b0164567a8af" containerName="main" probeResult="failure" output="Get \"https://10.133.0.64:8000/ready\": dial tcp 10.133.0.64:8000: connect: connection refused"
Oct 30 22:18:31.520135 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:31.520074    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-9f8aea/distinct-w5-9f8aea-kserve-5b564579cd-kmxzz"
Oct 30 22:18:31.604059 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:31.604019    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w0-dde1e8-9575db797-2whh8" podUID="35fcc100-bc0e-4c7a-9e34-aa06b86f288a" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:18:31.604353 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:31.604027    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-dde1e8-59fbfbdb67-vdb8w" podUID="40fb21f2-c4df-4bfd-b6e9-4198045570be" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:18:31.621342 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:31.621303    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b11e8b11-334f-4a9e-98c2-b0164567a8af-tls-certs\" (UniqueName: \"kubernetes.io/secret/b11e8b11-334f-4a9e-98c2-b0164567a8af-tls-certs\") pod \"b11e8b11-334f-4a9e-98c2-b0164567a8af\" (UID: \"b11e8b11-334f-4a9e-98c2-b0164567a8af\") "
Oct 30 22:18:31.621511 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:31.621375    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b11e8b11-334f-4a9e-98c2-b0164567a8af-dshm\" (UniqueName: \"kubernetes.io/empty-dir/b11e8b11-334f-4a9e-98c2-b0164567a8af-dshm\") pod \"b11e8b11-334f-4a9e-98c2-b0164567a8af\" (UID: \"b11e8b11-334f-4a9e-98c2-b0164567a8af\") "
Oct 30 22:18:31.621511 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:31.621398    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b11e8b11-334f-4a9e-98c2-b0164567a8af-home\" (UniqueName: \"kubernetes.io/empty-dir/b11e8b11-334f-4a9e-98c2-b0164567a8af-home\") pod \"b11e8b11-334f-4a9e-98c2-b0164567a8af\" (UID: \"b11e8b11-334f-4a9e-98c2-b0164567a8af\") "
Oct 30 22:18:31.621511 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:31.621433    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b11e8b11-334f-4a9e-98c2-b0164567a8af-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b11e8b11-334f-4a9e-98c2-b0164567a8af-model-cache\") pod \"b11e8b11-334f-4a9e-98c2-b0164567a8af\" (UID: \"b11e8b11-334f-4a9e-98c2-b0164567a8af\") "
Oct 30 22:18:31.621511 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:31.621477    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b11e8b11-334f-4a9e-98c2-b0164567a8af-kube-api-access-7fdfq\" (UniqueName: \"kubernetes.io/projected/b11e8b11-334f-4a9e-98c2-b0164567a8af-kube-api-access-7fdfq\") pod \"b11e8b11-334f-4a9e-98c2-b0164567a8af\" (UID: \"b11e8b11-334f-4a9e-98c2-b0164567a8af\") "
Oct 30 22:18:31.621807 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:31.621517    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b11e8b11-334f-4a9e-98c2-b0164567a8af-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b11e8b11-334f-4a9e-98c2-b0164567a8af-tmp-dir\") pod \"b11e8b11-334f-4a9e-98c2-b0164567a8af\" (UID: \"b11e8b11-334f-4a9e-98c2-b0164567a8af\") "
Oct 30 22:18:31.621807 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:31.621771    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b11e8b11-334f-4a9e-98c2-b0164567a8af-model-cache" pod "b11e8b11-334f-4a9e-98c2-b0164567a8af" (UID: "b11e8b11-334f-4a9e-98c2-b0164567a8af"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:18:31.621807 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:31.621758    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b11e8b11-334f-4a9e-98c2-b0164567a8af-home" pod "b11e8b11-334f-4a9e-98c2-b0164567a8af" (UID: "b11e8b11-334f-4a9e-98c2-b0164567a8af"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:18:31.621946 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:31.621838    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b11e8b11-334f-4a9e-98c2-b0164567a8af-tmp-dir" pod "b11e8b11-334f-4a9e-98c2-b0164567a8af" (UID: "b11e8b11-334f-4a9e-98c2-b0164567a8af"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:18:31.622018 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:31.622002    4473 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b11e8b11-334f-4a9e-98c2-b0164567a8af-home\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:18:31.622054 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:31.622023    4473 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b11e8b11-334f-4a9e-98c2-b0164567a8af-model-cache\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:18:31.622054 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:31.622033    4473 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b11e8b11-334f-4a9e-98c2-b0164567a8af-tmp-dir\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:18:31.623902 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:31.623875    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b11e8b11-334f-4a9e-98c2-b0164567a8af-dshm" pod "b11e8b11-334f-4a9e-98c2-b0164567a8af" (UID: "b11e8b11-334f-4a9e-98c2-b0164567a8af"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:18:31.624061 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:31.623910    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b11e8b11-334f-4a9e-98c2-b0164567a8af-kube-api-access-7fdfq" pod "b11e8b11-334f-4a9e-98c2-b0164567a8af" (UID: "b11e8b11-334f-4a9e-98c2-b0164567a8af"). InnerVolumeSpecName "kube-api-access-7fdfq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:18:31.624061 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:31.623990    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b11e8b11-334f-4a9e-98c2-b0164567a8af-tls-certs" pod "b11e8b11-334f-4a9e-98c2-b0164567a8af" (UID: "b11e8b11-334f-4a9e-98c2-b0164567a8af"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:18:31.723605 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:31.723558    4473 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7fdfq\" (UniqueName: \"kubernetes.io/projected/b11e8b11-334f-4a9e-98c2-b0164567a8af-kube-api-access-7fdfq\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:18:31.723605 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:31.723591    4473 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b11e8b11-334f-4a9e-98c2-b0164567a8af-tls-certs\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:18:31.723605 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:31.723612    4473 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b11e8b11-334f-4a9e-98c2-b0164567a8af-dshm\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:18:32.013441 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:32.013396    4473 generic.go:356] "Generic (PLEG): container finished" podID="b11e8b11-334f-4a9e-98c2-b0164567a8af" containerID="23e7478f0e398bb331ec0933335a676ae4bb1b251a6f49d13a25042b296a8bcb" exitCode=0
Oct 30 22:18:32.014039 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:32.014012    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-9f8aea/distinct-w5-9f8aea-kserve-5b564579cd-kmxzz"
Oct 30 22:18:32.014039 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:32.014024    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-9f8aea/distinct-w5-9f8aea-kserve-5b564579cd-kmxzz" event={"ID":"b11e8b11-334f-4a9e-98c2-b0164567a8af","Type":"ContainerDied","Data":"23e7478f0e398bb331ec0933335a676ae4bb1b251a6f49d13a25042b296a8bcb"}
Oct 30 22:18:32.014251 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:32.014084    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-9f8aea/distinct-w5-9f8aea-kserve-5b564579cd-kmxzz" event={"ID":"b11e8b11-334f-4a9e-98c2-b0164567a8af","Type":"ContainerDied","Data":"fff3bdc0d8e9077354df011d4bae82bbef91e131debca0c78a5848167cb457b3"}
Oct 30 22:18:32.014251 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:32.014113    4473 scope.go:122] "RemoveContainer" containerID="23e7478f0e398bb331ec0933335a676ae4bb1b251a6f49d13a25042b296a8bcb"
Oct 30 22:18:32.025084 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:32.025059    4473 scope.go:122] "RemoveContainer" containerID="23e7478f0e398bb331ec0933335a676ae4bb1b251a6f49d13a25042b296a8bcb"
Oct 30 22:18:32.025380 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:18:32.025361    4473 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"23e7478f0e398bb331ec0933335a676ae4bb1b251a6f49d13a25042b296a8bcb\": container with ID starting with 23e7478f0e398bb331ec0933335a676ae4bb1b251a6f49d13a25042b296a8bcb not found: ID does not exist" containerID="23e7478f0e398bb331ec0933335a676ae4bb1b251a6f49d13a25042b296a8bcb"
Oct 30 22:18:32.025430 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:32.025390    4473 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"23e7478f0e398bb331ec0933335a676ae4bb1b251a6f49d13a25042b296a8bcb"} err="failed to get container status \"23e7478f0e398bb331ec0933335a676ae4bb1b251a6f49d13a25042b296a8bcb\": rpc error: code = NotFound desc = could not find container \"23e7478f0e398bb331ec0933335a676ae4bb1b251a6f49d13a25042b296a8bcb\": container with ID starting with 23e7478f0e398bb331ec0933335a676ae4bb1b251a6f49d13a25042b296a8bcb not found: ID does not exist"
Oct 30 22:18:32.181355 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:32.181317    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w5-9f8aea/distinct-w5-9f8aea-kserve-5b564579cd-kmxzz"]
Oct 30 22:18:32.231650 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:32.231610    4473 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w5-9f8aea/distinct-w5-9f8aea-kserve-5b564579cd-kmxzz"]
Oct 30 22:18:32.357221 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:32.357149    4473 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b11e8b11-334f-4a9e-98c2-b0164567a8af" path="/var/lib/kubelet/pods/b11e8b11-334f-4a9e-98c2-b0164567a8af/volumes"
Oct 30 22:18:32.544039 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:32.544004    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7314-bb536d"]
Oct 30 22:18:32.547318 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:32.547287    4473 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7314-bb536d"]
Oct 30 22:18:33.019806 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:33.019752    4473 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/test-curl-7314-bb536d" podUID="55ac5838-c7d2-4ed9-9ab0-3923413b0392" containerName="test-curl-7314-bb536d" containerID="cri-o://ff5145ea0052ae2611086c7039ce5bdb4d84287ab95a83deb3eb58e5a0d91ea4" gracePeriod=2
Oct 30 22:18:33.071409 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:18:33.071365    4473 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7314-bb536d\" is forbidden: User \"system:node:ip-10-0-140-18.us-west-1.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-140-18.us-west-1.compute.internal' and this object" podUID="55ac5838-c7d2-4ed9-9ab0-3923413b0392" pod="opendatahub/test-curl-7314-bb536d"
Oct 30 22:18:34.405778 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:18:34.405734    4473 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7314-bb536d\" is forbidden: User \"system:node:ip-10-0-140-18.us-west-1.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-140-18.us-west-1.compute.internal' and this object" podUID="55ac5838-c7d2-4ed9-9ab0-3923413b0392" pod="opendatahub/test-curl-7314-bb536d"
Oct 30 22:18:34.652257 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:34.652225    4473 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-9383b377-tlnrt"]
Oct 30 22:18:34.652735 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:34.652722    4473 cpu_manager.go:414] "containerMap: removing container" podUID="b11e8b11-334f-4a9e-98c2-b0164567a8af" containerName="main"
Oct 30 22:18:34.652784 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:34.652737    4473 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b11e8b11-334f-4a9e-98c2-b0164567a8af" containerName="main"
Oct 30 22:18:34.652784 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:34.652763    4473 cpu_manager.go:414] "containerMap: removing container" podUID="55ac5838-c7d2-4ed9-9ab0-3923413b0392" containerName="test-curl-7314-bb536d"
Oct 30 22:18:34.652784 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:34.652768    4473 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="55ac5838-c7d2-4ed9-9ab0-3923413b0392" containerName="test-curl-7314-bb536d"
Oct 30 22:18:34.652885 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:34.652836    4473 memory_manager.go:365] "RemoveStaleState removing state" podUID="b11e8b11-334f-4a9e-98c2-b0164567a8af" containerName="main"
Oct 30 22:18:34.652885 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:34.652848    4473 memory_manager.go:365] "RemoveStaleState removing state" podUID="893a5438-3b54-4d77-8675-00260980c0f6" containerName="istio-proxy"
Oct 30 22:18:34.652885 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:34.652854    4473 memory_manager.go:365] "RemoveStaleState removing state" podUID="55ac5838-c7d2-4ed9-9ab0-3923413b0392" containerName="test-curl-7314-bb536d"
Oct 30 22:18:34.657565 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:34.657512    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9383b377-tlnrt"
Oct 30 22:18:34.717224 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:34.717190    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-9383b377-tlnrt"]
Oct 30 22:18:34.761081 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:34.761055    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 22:18:34.761264 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:34.761135    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-d786g\""
Oct 30 22:18:34.761588 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:34.761567    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/0b139aed-161c-425c-a63c-59441bcdaa54-maas-api-xks-ca\") pod \"maas-api-revoke-sub-9383b377-tlnrt\" (UID: \"0b139aed-161c-425c-a63c-59441bcdaa54\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9383b377-tlnrt"
Oct 30 22:18:34.761646 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:34.761602    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dzkj2\" (UniqueName: \"kubernetes.io/projected/0b139aed-161c-425c-a63c-59441bcdaa54-kube-api-access-dzkj2\") pod \"maas-api-revoke-sub-9383b377-tlnrt\" (UID: \"0b139aed-161c-425c-a63c-59441bcdaa54\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9383b377-tlnrt"
Oct 30 22:18:34.761646 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:34.761627    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/0b139aed-161c-425c-a63c-59441bcdaa54-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-9383b377-tlnrt\" (UID: \"0b139aed-161c-425c-a63c-59441bcdaa54\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9383b377-tlnrt"
Oct 30 22:18:34.864785 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:34.863418    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/0b139aed-161c-425c-a63c-59441bcdaa54-maas-api-xks-ca\") pod \"maas-api-revoke-sub-9383b377-tlnrt\" (UID: \"0b139aed-161c-425c-a63c-59441bcdaa54\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9383b377-tlnrt"
Oct 30 22:18:34.864785 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:34.863481    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dzkj2\" (UniqueName: \"kubernetes.io/projected/0b139aed-161c-425c-a63c-59441bcdaa54-kube-api-access-dzkj2\") pod \"maas-api-revoke-sub-9383b377-tlnrt\" (UID: \"0b139aed-161c-425c-a63c-59441bcdaa54\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9383b377-tlnrt"
Oct 30 22:18:34.864785 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:34.863523    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/0b139aed-161c-425c-a63c-59441bcdaa54-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-9383b377-tlnrt\" (UID: \"0b139aed-161c-425c-a63c-59441bcdaa54\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9383b377-tlnrt"
Oct 30 22:18:34.864785 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:34.864441    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/0b139aed-161c-425c-a63c-59441bcdaa54-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-9383b377-tlnrt\" (UID: \"0b139aed-161c-425c-a63c-59441bcdaa54\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9383b377-tlnrt"
Oct 30 22:18:34.867496 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:34.867468    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/0b139aed-161c-425c-a63c-59441bcdaa54-maas-api-xks-ca\") pod \"maas-api-revoke-sub-9383b377-tlnrt\" (UID: \"0b139aed-161c-425c-a63c-59441bcdaa54\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9383b377-tlnrt"
Oct 30 22:18:34.923690 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:34.923612    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dzkj2\" (UniqueName: \"kubernetes.io/projected/0b139aed-161c-425c-a63c-59441bcdaa54-kube-api-access-dzkj2\") pod \"maas-api-revoke-sub-9383b377-tlnrt\" (UID: \"0b139aed-161c-425c-a63c-59441bcdaa54\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9383b377-tlnrt"
Oct 30 22:18:34.971042 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:34.971009    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9383b377-tlnrt"
Oct 30 22:18:35.249616 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:35.249587    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7314-bb536d"
Oct 30 22:18:35.345197 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:35.345169    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-9383b377-tlnrt"]
Oct 30 22:18:35.345858 ip-10-0-140-18 kubenswrapper[4473]: W1030 22:18:35.345822    4473 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b139aed_161c_425c_a63c_59441bcdaa54.slice/crio-5648a5164b0d891847ce22909f1d410bf71f1c8fa6b644b73818ba44f16d19e6 WatchSource:0}: Error finding container 5648a5164b0d891847ce22909f1d410bf71f1c8fa6b644b73818ba44f16d19e6: Status 404 returned error can't find the container with id 5648a5164b0d891847ce22909f1d410bf71f1c8fa6b644b73818ba44f16d19e6
Oct 30 22:18:35.370788 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:35.370754    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/55ac5838-c7d2-4ed9-9ab0-3923413b0392-kube-api-access-vnvgt\" (UniqueName: \"kubernetes.io/projected/55ac5838-c7d2-4ed9-9ab0-3923413b0392-kube-api-access-vnvgt\") pod \"55ac5838-c7d2-4ed9-9ab0-3923413b0392\" (UID: \"55ac5838-c7d2-4ed9-9ab0-3923413b0392\") "
Oct 30 22:18:35.372892 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:35.372869    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/55ac5838-c7d2-4ed9-9ab0-3923413b0392-kube-api-access-vnvgt" pod "55ac5838-c7d2-4ed9-9ab0-3923413b0392" (UID: "55ac5838-c7d2-4ed9-9ab0-3923413b0392"). InnerVolumeSpecName "kube-api-access-vnvgt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:18:35.472303 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:35.472213    4473 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vnvgt\" (UniqueName: \"kubernetes.io/projected/55ac5838-c7d2-4ed9-9ab0-3923413b0392-kube-api-access-vnvgt\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:18:36.046876 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:36.046792    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9383b377-tlnrt" event={"ID":"0b139aed-161c-425c-a63c-59441bcdaa54","Type":"ContainerStarted","Data":"249747718be503b6a204625d92a84b8de2c4e38cff4b25d6ffc3df8de037595f"}
Oct 30 22:18:36.046876 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:36.046850    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9383b377-tlnrt" event={"ID":"0b139aed-161c-425c-a63c-59441bcdaa54","Type":"ContainerStarted","Data":"5648a5164b0d891847ce22909f1d410bf71f1c8fa6b644b73818ba44f16d19e6"}
Oct 30 22:18:36.048177 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:36.048152    4473 generic.go:356] "Generic (PLEG): container finished" podID="55ac5838-c7d2-4ed9-9ab0-3923413b0392" containerID="ff5145ea0052ae2611086c7039ce5bdb4d84287ab95a83deb3eb58e5a0d91ea4" exitCode=137
Oct 30 22:18:36.048298 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:36.048185    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7314-bb536d"
Oct 30 22:18:36.048298 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:36.048206    4473 scope.go:122] "RemoveContainer" containerID="ff5145ea0052ae2611086c7039ce5bdb4d84287ab95a83deb3eb58e5a0d91ea4"
Oct 30 22:18:36.066846 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:36.066825    4473 scope.go:122] "RemoveContainer" containerID="ff5145ea0052ae2611086c7039ce5bdb4d84287ab95a83deb3eb58e5a0d91ea4"
Oct 30 22:18:36.067180 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:18:36.067161    4473 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ff5145ea0052ae2611086c7039ce5bdb4d84287ab95a83deb3eb58e5a0d91ea4\": container with ID starting with ff5145ea0052ae2611086c7039ce5bdb4d84287ab95a83deb3eb58e5a0d91ea4 not found: ID does not exist" containerID="ff5145ea0052ae2611086c7039ce5bdb4d84287ab95a83deb3eb58e5a0d91ea4"
Oct 30 22:18:36.067256 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:36.067193    4473 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ff5145ea0052ae2611086c7039ce5bdb4d84287ab95a83deb3eb58e5a0d91ea4"} err="failed to get container status \"ff5145ea0052ae2611086c7039ce5bdb4d84287ab95a83deb3eb58e5a0d91ea4\": rpc error: code = NotFound desc = could not find container \"ff5145ea0052ae2611086c7039ce5bdb4d84287ab95a83deb3eb58e5a0d91ea4\": container with ID starting with ff5145ea0052ae2611086c7039ce5bdb4d84287ab95a83deb3eb58e5a0d91ea4 not found: ID does not exist"
Oct 30 22:18:36.166981 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:36.166907    4473 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9383b377-tlnrt" podStartSLOduration=2.16689306 podStartE2EDuration="2.16689306s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:18:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:18:36.16375742 +0000 UTC m=+2161.937125913" watchObservedRunningTime="2026-10-30 22:18:36.16689306 +0000 UTC m=+2161.940261547"
Oct 30 22:18:36.215791 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:18:36.215738    4473 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7314-bb536d\" is forbidden: User \"system:node:ip-10-0-140-18.us-west-1.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-140-18.us-west-1.compute.internal' and this object" podUID="55ac5838-c7d2-4ed9-9ab0-3923413b0392" pod="opendatahub/test-curl-7314-bb536d"
Oct 30 22:18:36.357048 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:36.356972    4473 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="55ac5838-c7d2-4ed9-9ab0-3923413b0392" path="/var/lib/kubelet/pods/55ac5838-c7d2-4ed9-9ab0-3923413b0392/volumes"
Oct 30 22:18:37.055713 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:37.055680    4473 generic.go:356] "Generic (PLEG): container finished" podID="0b139aed-161c-425c-a63c-59441bcdaa54" containerID="249747718be503b6a204625d92a84b8de2c4e38cff4b25d6ffc3df8de037595f" exitCode=0
Oct 30 22:18:37.056222 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:37.055771    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9383b377-tlnrt" event={"ID":"0b139aed-161c-425c-a63c-59441bcdaa54","Type":"ContainerDied","Data":"249747718be503b6a204625d92a84b8de2c4e38cff4b25d6ffc3df8de037595f"}
Oct 30 22:18:37.307892 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:37.307807    4473 scope.go:122] "RemoveContainer" containerID="1c86b6a4c299895072f9e0fa08affdb99c0553a35b96a9364a9b4f069eaee262"
Oct 30 22:18:38.213213 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:38.213188    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9383b377-tlnrt"
Oct 30 22:18:38.303393 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:38.303360    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0b139aed-161c-425c-a63c-59441bcdaa54-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/0b139aed-161c-425c-a63c-59441bcdaa54-maas-api-openshift-ca\") pod \"0b139aed-161c-425c-a63c-59441bcdaa54\" (UID: \"0b139aed-161c-425c-a63c-59441bcdaa54\") "
Oct 30 22:18:38.303606 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:38.303449    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0b139aed-161c-425c-a63c-59441bcdaa54-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/0b139aed-161c-425c-a63c-59441bcdaa54-maas-api-xks-ca\") pod \"0b139aed-161c-425c-a63c-59441bcdaa54\" (UID: \"0b139aed-161c-425c-a63c-59441bcdaa54\") "
Oct 30 22:18:38.303686 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:38.303612    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0b139aed-161c-425c-a63c-59441bcdaa54-kube-api-access-dzkj2\" (UniqueName: \"kubernetes.io/projected/0b139aed-161c-425c-a63c-59441bcdaa54-kube-api-access-dzkj2\") pod \"0b139aed-161c-425c-a63c-59441bcdaa54\" (UID: \"0b139aed-161c-425c-a63c-59441bcdaa54\") "
Oct 30 22:18:38.303782 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:38.303758    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0b139aed-161c-425c-a63c-59441bcdaa54-maas-api-openshift-ca" pod "0b139aed-161c-425c-a63c-59441bcdaa54" (UID: "0b139aed-161c-425c-a63c-59441bcdaa54"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:18:38.304104 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:38.304082    4473 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/0b139aed-161c-425c-a63c-59441bcdaa54-maas-api-openshift-ca\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:18:38.305757 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:38.305737    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0b139aed-161c-425c-a63c-59441bcdaa54-maas-api-xks-ca" pod "0b139aed-161c-425c-a63c-59441bcdaa54" (UID: "0b139aed-161c-425c-a63c-59441bcdaa54"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:18:38.305833 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:38.305770    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0b139aed-161c-425c-a63c-59441bcdaa54-kube-api-access-dzkj2" pod "0b139aed-161c-425c-a63c-59441bcdaa54" (UID: "0b139aed-161c-425c-a63c-59441bcdaa54"). InnerVolumeSpecName "kube-api-access-dzkj2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:18:38.405383 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:38.405342    4473 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/0b139aed-161c-425c-a63c-59441bcdaa54-maas-api-xks-ca\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:18:38.405383 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:38.405377    4473 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dzkj2\" (UniqueName: \"kubernetes.io/projected/0b139aed-161c-425c-a63c-59441bcdaa54-kube-api-access-dzkj2\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:18:38.506054 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:38.506009    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-9f8aea-df9ff57d7-k7mlk" podUID="7665007f-2e07-47f7-ac42-82dfd53d85a1" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:18:39.072102 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:39.072071    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9383b377-tlnrt"
Oct 30 22:18:39.072102 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:39.072090    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9383b377-tlnrt" event={"ID":"0b139aed-161c-425c-a63c-59441bcdaa54","Type":"ContainerDied","Data":"5648a5164b0d891847ce22909f1d410bf71f1c8fa6b644b73818ba44f16d19e6"}
Oct 30 22:18:39.072314 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:39.072143    4473 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5648a5164b0d891847ce22909f1d410bf71f1c8fa6b644b73818ba44f16d19e6"
Oct 30 22:18:41.604195 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:41.604148    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w0-dde1e8-9575db797-2whh8" podUID="35fcc100-bc0e-4c7a-9e34-aa06b86f288a" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:18:41.604563 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:41.604163    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-dde1e8-59fbfbdb67-vdb8w" podUID="40fb21f2-c4df-4bfd-b6e9-4198045570be" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:18:43.520970 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:43.520862    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-dde1e8-9575db797-2whh8"
Oct 30 22:18:43.613448 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:43.613422    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-dde1e8-59fbfbdb67-vdb8w"
Oct 30 22:18:43.665840 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:43.665801    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/35fcc100-bc0e-4c7a-9e34-aa06b86f288a-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/35fcc100-bc0e-4c7a-9e34-aa06b86f288a-plugins-config-volume\") pod \"35fcc100-bc0e-4c7a-9e34-aa06b86f288a\" (UID: \"35fcc100-bc0e-4c7a-9e34-aa06b86f288a\") "
Oct 30 22:18:43.666030 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:43.665955    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/35fcc100-bc0e-4c7a-9e34-aa06b86f288a-kube-api-access-jcv2d\" (UniqueName: \"kubernetes.io/projected/35fcc100-bc0e-4c7a-9e34-aa06b86f288a-kube-api-access-jcv2d\") pod \"35fcc100-bc0e-4c7a-9e34-aa06b86f288a\" (UID: \"35fcc100-bc0e-4c7a-9e34-aa06b86f288a\") "
Oct 30 22:18:43.666229 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:43.666209    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/35fcc100-bc0e-4c7a-9e34-aa06b86f288a-plugins-config-volume" pod "35fcc100-bc0e-4c7a-9e34-aa06b86f288a" (UID: "35fcc100-bc0e-4c7a-9e34-aa06b86f288a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:18:43.666409 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:43.666385    4473 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/35fcc100-bc0e-4c7a-9e34-aa06b86f288a-plugins-config-volume\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:18:43.668184 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:43.668163    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/35fcc100-bc0e-4c7a-9e34-aa06b86f288a-kube-api-access-jcv2d" pod "35fcc100-bc0e-4c7a-9e34-aa06b86f288a" (UID: "35fcc100-bc0e-4c7a-9e34-aa06b86f288a"). InnerVolumeSpecName "kube-api-access-jcv2d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:18:43.767147 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:43.767115    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/40fb21f2-c4df-4bfd-b6e9-4198045570be-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/40fb21f2-c4df-4bfd-b6e9-4198045570be-plugins-config-volume\") pod \"40fb21f2-c4df-4bfd-b6e9-4198045570be\" (UID: \"40fb21f2-c4df-4bfd-b6e9-4198045570be\") "
Oct 30 22:18:43.767351 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:43.767253    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/40fb21f2-c4df-4bfd-b6e9-4198045570be-kube-api-access-njdz2\" (UniqueName: \"kubernetes.io/projected/40fb21f2-c4df-4bfd-b6e9-4198045570be-kube-api-access-njdz2\") pod \"40fb21f2-c4df-4bfd-b6e9-4198045570be\" (UID: \"40fb21f2-c4df-4bfd-b6e9-4198045570be\") "
Oct 30 22:18:43.767565 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:43.767540    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/40fb21f2-c4df-4bfd-b6e9-4198045570be-plugins-config-volume" pod "40fb21f2-c4df-4bfd-b6e9-4198045570be" (UID: "40fb21f2-c4df-4bfd-b6e9-4198045570be"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:18:43.767683 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:43.767667    4473 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/40fb21f2-c4df-4bfd-b6e9-4198045570be-plugins-config-volume\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:18:43.767749 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:43.767685    4473 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jcv2d\" (UniqueName: \"kubernetes.io/projected/35fcc100-bc0e-4c7a-9e34-aa06b86f288a-kube-api-access-jcv2d\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:18:43.769496 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:43.769468    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/40fb21f2-c4df-4bfd-b6e9-4198045570be-kube-api-access-njdz2" pod "40fb21f2-c4df-4bfd-b6e9-4198045570be" (UID: "40fb21f2-c4df-4bfd-b6e9-4198045570be"). InnerVolumeSpecName "kube-api-access-njdz2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:18:43.868912 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:43.868877    4473 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-njdz2\" (UniqueName: \"kubernetes.io/projected/40fb21f2-c4df-4bfd-b6e9-4198045570be-kube-api-access-njdz2\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:18:44.101255 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:44.101220    4473 generic.go:356] "Generic (PLEG): container finished" podID="35fcc100-bc0e-4c7a-9e34-aa06b86f288a" containerID="212ee3847d8144b97dc7b6de30a8a3ff313835f090a9da740d069666b6c9fd2b" exitCode=0
Oct 30 22:18:44.101455 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:44.101270    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-dde1e8-9575db797-2whh8"
Oct 30 22:18:44.101455 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:44.101303    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-dde1e8-9575db797-2whh8" event={"ID":"35fcc100-bc0e-4c7a-9e34-aa06b86f288a","Type":"ContainerDied","Data":"212ee3847d8144b97dc7b6de30a8a3ff313835f090a9da740d069666b6c9fd2b"}
Oct 30 22:18:44.101455 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:44.101369    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-dde1e8-9575db797-2whh8" event={"ID":"35fcc100-bc0e-4c7a-9e34-aa06b86f288a","Type":"ContainerDied","Data":"8de58918b86455086f1fcb0c2d5b912d24b67e1c6c42104dcef4fc8889e25a5c"}
Oct 30 22:18:44.101455 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:44.101396    4473 scope.go:122] "RemoveContainer" containerID="212ee3847d8144b97dc7b6de30a8a3ff313835f090a9da740d069666b6c9fd2b"
Oct 30 22:18:44.102768 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:44.102743    4473 generic.go:356] "Generic (PLEG): container finished" podID="40fb21f2-c4df-4bfd-b6e9-4198045570be" containerID="1a7b98080bc79bb81666c15e653c237883b2d9eb22ade815b772790a02231b1a" exitCode=0
Oct 30 22:18:44.102897 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:44.102796    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-dde1e8-59fbfbdb67-vdb8w"
Oct 30 22:18:44.102897 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:44.102833    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-dde1e8-59fbfbdb67-vdb8w" event={"ID":"40fb21f2-c4df-4bfd-b6e9-4198045570be","Type":"ContainerDied","Data":"1a7b98080bc79bb81666c15e653c237883b2d9eb22ade815b772790a02231b1a"}
Oct 30 22:18:44.102897 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:44.102881    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-dde1e8-59fbfbdb67-vdb8w" event={"ID":"40fb21f2-c4df-4bfd-b6e9-4198045570be","Type":"ContainerDied","Data":"0687e3190b109823764c9a2f0aec2e75ccb748c6c80c96afb1b8e57494695494"}
Oct 30 22:18:44.112181 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:44.112151    4473 scope.go:122] "RemoveContainer" containerID="212ee3847d8144b97dc7b6de30a8a3ff313835f090a9da740d069666b6c9fd2b"
Oct 30 22:18:44.112574 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:18:44.112544    4473 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"212ee3847d8144b97dc7b6de30a8a3ff313835f090a9da740d069666b6c9fd2b\": container with ID starting with 212ee3847d8144b97dc7b6de30a8a3ff313835f090a9da740d069666b6c9fd2b not found: ID does not exist" containerID="212ee3847d8144b97dc7b6de30a8a3ff313835f090a9da740d069666b6c9fd2b"
Oct 30 22:18:44.112678 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:44.112587    4473 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"212ee3847d8144b97dc7b6de30a8a3ff313835f090a9da740d069666b6c9fd2b"} err="failed to get container status \"212ee3847d8144b97dc7b6de30a8a3ff313835f090a9da740d069666b6c9fd2b\": rpc error: code = NotFound desc = could not find container \"212ee3847d8144b97dc7b6de30a8a3ff313835f090a9da740d069666b6c9fd2b\": container with ID starting with 212ee3847d8144b97dc7b6de30a8a3ff313835f090a9da740d069666b6c9fd2b not found: ID does not exist"
Oct 30 22:18:44.112678 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:44.112610    4473 scope.go:122] "RemoveContainer" containerID="1a7b98080bc79bb81666c15e653c237883b2d9eb22ade815b772790a02231b1a"
Oct 30 22:18:44.121752 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:44.121730    4473 scope.go:122] "RemoveContainer" containerID="1a7b98080bc79bb81666c15e653c237883b2d9eb22ade815b772790a02231b1a"
Oct 30 22:18:44.122104 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:18:44.122084    4473 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1a7b98080bc79bb81666c15e653c237883b2d9eb22ade815b772790a02231b1a\": container with ID starting with 1a7b98080bc79bb81666c15e653c237883b2d9eb22ade815b772790a02231b1a not found: ID does not exist" containerID="1a7b98080bc79bb81666c15e653c237883b2d9eb22ade815b772790a02231b1a"
Oct 30 22:18:44.122185 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:44.122114    4473 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1a7b98080bc79bb81666c15e653c237883b2d9eb22ade815b772790a02231b1a"} err="failed to get container status \"1a7b98080bc79bb81666c15e653c237883b2d9eb22ade815b772790a02231b1a\": rpc error: code = NotFound desc = could not find container \"1a7b98080bc79bb81666c15e653c237883b2d9eb22ade815b772790a02231b1a\": container with ID starting with 1a7b98080bc79bb81666c15e653c237883b2d9eb22ade815b772790a02231b1a not found: ID does not exist"
Oct 30 22:18:44.267894 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:44.267859    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-dde1e8-59fbfbdb67-vdb8w"]
Oct 30 22:18:44.271591 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:44.271561    4473 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-dde1e8-59fbfbdb67-vdb8w"]
Oct 30 22:18:44.356144 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:44.356111    4473 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="40fb21f2-c4df-4bfd-b6e9-4198045570be" path="/var/lib/kubelet/pods/40fb21f2-c4df-4bfd-b6e9-4198045570be/volumes"
Oct 30 22:18:44.381683 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:44.381599    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-dde1e8-9575db797-2whh8"]
Oct 30 22:18:44.385141 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:44.385109    4473 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-dde1e8-9575db797-2whh8"]
Oct 30 22:18:44.622511 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:44.622475    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-242ddaf0-576ccc4d89-28sq2"]
Oct 30 22:18:44.732585 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:44.732493    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-242ddaf0-7b7b974bc-5rsm9"]
Oct 30 22:18:46.356768 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:46.356737    4473 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="35fcc100-bc0e-4c7a-9e34-aa06b86f288a" path="/var/lib/kubelet/pods/35fcc100-bc0e-4c7a-9e34-aa06b86f288a/volumes"
Oct 30 22:18:48.504481 ip-10-0-140-18 kubenswrapper[4473]: W1030 22:18:48.504452    4473 logging.go:55] [core] [Channel #5048 SubChannel #5049]grpc: addrConn.createTransport failed to connect to {Addr: "10.133.0.53:50052", ServerName: "10.133.0.53:50052", BalancerAttributes: {"<%!p(pickfirstleaf.managedByPickfirstKeyType={})>": "<%!p(bool=true)>" }}. Err: connection error: desc = "transport: Error while dialing: dial tcp 10.133.0.53:50052: connect: connection refused"
Oct 30 22:18:48.678260 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:48.678201    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-9f8aea-df9ff57d7-k7mlk"
Oct 30 22:18:48.742447 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:48.742414    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7665007f-2e07-47f7-ac42-82dfd53d85a1-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7665007f-2e07-47f7-ac42-82dfd53d85a1-plugins-config-volume\") pod \"7665007f-2e07-47f7-ac42-82dfd53d85a1\" (UID: \"7665007f-2e07-47f7-ac42-82dfd53d85a1\") "
Oct 30 22:18:48.742624 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:48.742550    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7665007f-2e07-47f7-ac42-82dfd53d85a1-kube-api-access-d6bw8\" (UniqueName: \"kubernetes.io/projected/7665007f-2e07-47f7-ac42-82dfd53d85a1-kube-api-access-d6bw8\") pod \"7665007f-2e07-47f7-ac42-82dfd53d85a1\" (UID: \"7665007f-2e07-47f7-ac42-82dfd53d85a1\") "
Oct 30 22:18:48.742772 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:48.742752    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7665007f-2e07-47f7-ac42-82dfd53d85a1-plugins-config-volume" pod "7665007f-2e07-47f7-ac42-82dfd53d85a1" (UID: "7665007f-2e07-47f7-ac42-82dfd53d85a1"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:18:48.744939 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:48.744896    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7665007f-2e07-47f7-ac42-82dfd53d85a1-kube-api-access-d6bw8" pod "7665007f-2e07-47f7-ac42-82dfd53d85a1" (UID: "7665007f-2e07-47f7-ac42-82dfd53d85a1"). InnerVolumeSpecName "kube-api-access-d6bw8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:18:48.843773 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:48.843741    4473 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7665007f-2e07-47f7-ac42-82dfd53d85a1-plugins-config-volume\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:18:48.843773 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:48.843768    4473 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d6bw8\" (UniqueName: \"kubernetes.io/projected/7665007f-2e07-47f7-ac42-82dfd53d85a1-kube-api-access-d6bw8\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:18:49.136869 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:49.136833    4473 generic.go:356] "Generic (PLEG): container finished" podID="7665007f-2e07-47f7-ac42-82dfd53d85a1" containerID="533a86cb750ea2e47f0f08fd67a09ed43ebe877cddb816f28ed25be603352c5c" exitCode=0
Oct 30 22:18:49.137099 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:49.136882    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-9f8aea-df9ff57d7-k7mlk"
Oct 30 22:18:49.137099 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:49.136911    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-9f8aea-df9ff57d7-k7mlk" event={"ID":"7665007f-2e07-47f7-ac42-82dfd53d85a1","Type":"ContainerDied","Data":"533a86cb750ea2e47f0f08fd67a09ed43ebe877cddb816f28ed25be603352c5c"}
Oct 30 22:18:49.137099 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:49.137000    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-9f8aea-df9ff57d7-k7mlk" event={"ID":"7665007f-2e07-47f7-ac42-82dfd53d85a1","Type":"ContainerDied","Data":"c97dc21c154a651d5dee7eb506afd85abb479c66b8c64f45011dfee87975df2c"}
Oct 30 22:18:49.137099 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:49.137016    4473 scope.go:122] "RemoveContainer" containerID="533a86cb750ea2e47f0f08fd67a09ed43ebe877cddb816f28ed25be603352c5c"
Oct 30 22:18:49.146965 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:49.146940    4473 scope.go:122] "RemoveContainer" containerID="533a86cb750ea2e47f0f08fd67a09ed43ebe877cddb816f28ed25be603352c5c"
Oct 30 22:18:49.147308 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:18:49.147287    4473 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"533a86cb750ea2e47f0f08fd67a09ed43ebe877cddb816f28ed25be603352c5c\": container with ID starting with 533a86cb750ea2e47f0f08fd67a09ed43ebe877cddb816f28ed25be603352c5c not found: ID does not exist" containerID="533a86cb750ea2e47f0f08fd67a09ed43ebe877cddb816f28ed25be603352c5c"
Oct 30 22:18:49.147374 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:49.147326    4473 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"533a86cb750ea2e47f0f08fd67a09ed43ebe877cddb816f28ed25be603352c5c"} err="failed to get container status \"533a86cb750ea2e47f0f08fd67a09ed43ebe877cddb816f28ed25be603352c5c\": rpc error: code = NotFound desc = could not find container \"533a86cb750ea2e47f0f08fd67a09ed43ebe877cddb816f28ed25be603352c5c\": container with ID starting with 533a86cb750ea2e47f0f08fd67a09ed43ebe877cddb816f28ed25be603352c5c not found: ID does not exist"
Oct 30 22:18:49.305803 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:49.305767    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w5-9f8aea-df9ff57d7-k7mlk"]
Oct 30 22:18:49.356492 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:49.356453    4473 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w5-9f8aea-df9ff57d7-k7mlk"]
Oct 30 22:18:49.504611 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:49.504522    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-9f8aea-df9ff57d7-k7mlk" podUID="7665007f-2e07-47f7-ac42-82dfd53d85a1" containerName="payload-pre-processing" probeResult="failure" output="timeout: failed to connect service \"10.133.0.53:50052\" within 1s: context deadline exceeded"
Oct 30 22:18:49.623225 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:49.623165    4473 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-242ddaf0-576ccc4d89-28sq2" podUID="cd728188-2246-4f1d-8f18-ec351a96b935" containerName="payload-processing" containerID="cri-o://2aa491f0eb496338cbc817794a6e52e9bc7227a25da76367bff6045f176d81bb" gracePeriod=25
Oct 30 22:18:49.733468 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:49.733422    4473 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-242ddaf0-7b7b974bc-5rsm9" podUID="706bd83a-953d-4b2e-8f1c-e51bf4343a19" containerName="payload-pre-processing" containerID="cri-o://79868169be9b695577a671c7b4eac2561bce79cfa7f25fbd94d5c89f4bb87cb0" gracePeriod=25
Oct 30 22:18:49.899630 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:49.899430    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-242ddaf0-576ccc4d89-28sq2"
Oct 30 22:18:49.958222 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:49.958187    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cd728188-2246-4f1d-8f18-ec351a96b935-kube-api-access-v7vls\" (UniqueName: \"kubernetes.io/projected/cd728188-2246-4f1d-8f18-ec351a96b935-kube-api-access-v7vls\") pod \"cd728188-2246-4f1d-8f18-ec351a96b935\" (UID: \"cd728188-2246-4f1d-8f18-ec351a96b935\") "
Oct 30 22:18:49.958435 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:49.958320    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/cd728188-2246-4f1d-8f18-ec351a96b935-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cd728188-2246-4f1d-8f18-ec351a96b935-plugins-config-volume\") pod \"cd728188-2246-4f1d-8f18-ec351a96b935\" (UID: \"cd728188-2246-4f1d-8f18-ec351a96b935\") "
Oct 30 22:18:49.959008 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:49.958975    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cd728188-2246-4f1d-8f18-ec351a96b935-plugins-config-volume" pod "cd728188-2246-4f1d-8f18-ec351a96b935" (UID: "cd728188-2246-4f1d-8f18-ec351a96b935"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:18:49.962534 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:49.962502    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cd728188-2246-4f1d-8f18-ec351a96b935-kube-api-access-v7vls" pod "cd728188-2246-4f1d-8f18-ec351a96b935" (UID: "cd728188-2246-4f1d-8f18-ec351a96b935"). InnerVolumeSpecName "kube-api-access-v7vls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:18:49.991913 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:49.991880    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-242ddaf0-7b7b974bc-5rsm9"
Oct 30 22:18:50.059621 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:50.059539    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/706bd83a-953d-4b2e-8f1c-e51bf4343a19-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/706bd83a-953d-4b2e-8f1c-e51bf4343a19-plugins-config-volume\") pod \"706bd83a-953d-4b2e-8f1c-e51bf4343a19\" (UID: \"706bd83a-953d-4b2e-8f1c-e51bf4343a19\") "
Oct 30 22:18:50.059621 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:50.059582    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/706bd83a-953d-4b2e-8f1c-e51bf4343a19-kube-api-access-czbtn\" (UniqueName: \"kubernetes.io/projected/706bd83a-953d-4b2e-8f1c-e51bf4343a19-kube-api-access-czbtn\") pod \"706bd83a-953d-4b2e-8f1c-e51bf4343a19\" (UID: \"706bd83a-953d-4b2e-8f1c-e51bf4343a19\") "
Oct 30 22:18:50.059958 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:50.059908    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/706bd83a-953d-4b2e-8f1c-e51bf4343a19-plugins-config-volume" pod "706bd83a-953d-4b2e-8f1c-e51bf4343a19" (UID: "706bd83a-953d-4b2e-8f1c-e51bf4343a19"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:18:50.059958 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:50.059953    4473 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cd728188-2246-4f1d-8f18-ec351a96b935-plugins-config-volume\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:18:50.060083 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:50.059975    4473 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v7vls\" (UniqueName: \"kubernetes.io/projected/cd728188-2246-4f1d-8f18-ec351a96b935-kube-api-access-v7vls\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:18:50.062091 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:50.062064    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/706bd83a-953d-4b2e-8f1c-e51bf4343a19-kube-api-access-czbtn" pod "706bd83a-953d-4b2e-8f1c-e51bf4343a19" (UID: "706bd83a-953d-4b2e-8f1c-e51bf4343a19"). InnerVolumeSpecName "kube-api-access-czbtn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:18:50.144961 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:50.144886    4473 generic.go:356] "Generic (PLEG): container finished" podID="cd728188-2246-4f1d-8f18-ec351a96b935" containerID="2aa491f0eb496338cbc817794a6e52e9bc7227a25da76367bff6045f176d81bb" exitCode=0
Oct 30 22:18:50.144961 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:50.144947    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-242ddaf0-576ccc4d89-28sq2"
Oct 30 22:18:50.144961 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:50.144954    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-242ddaf0-576ccc4d89-28sq2" event={"ID":"cd728188-2246-4f1d-8f18-ec351a96b935","Type":"ContainerDied","Data":"2aa491f0eb496338cbc817794a6e52e9bc7227a25da76367bff6045f176d81bb"}
Oct 30 22:18:50.145280 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:50.145013    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-242ddaf0-576ccc4d89-28sq2" event={"ID":"cd728188-2246-4f1d-8f18-ec351a96b935","Type":"ContainerDied","Data":"4c3767f45501a73ce7c0125c76badb300c33fae7d6fb18d4d83ae3bf04580e33"}
Oct 30 22:18:50.145280 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:50.145036    4473 scope.go:122] "RemoveContainer" containerID="2aa491f0eb496338cbc817794a6e52e9bc7227a25da76367bff6045f176d81bb"
Oct 30 22:18:50.148077 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:50.148047    4473 generic.go:356] "Generic (PLEG): container finished" podID="706bd83a-953d-4b2e-8f1c-e51bf4343a19" containerID="79868169be9b695577a671c7b4eac2561bce79cfa7f25fbd94d5c89f4bb87cb0" exitCode=0
Oct 30 22:18:50.148228 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:50.148099    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-242ddaf0-7b7b974bc-5rsm9"
Oct 30 22:18:50.148228 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:50.148131    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-242ddaf0-7b7b974bc-5rsm9" event={"ID":"706bd83a-953d-4b2e-8f1c-e51bf4343a19","Type":"ContainerDied","Data":"79868169be9b695577a671c7b4eac2561bce79cfa7f25fbd94d5c89f4bb87cb0"}
Oct 30 22:18:50.148228 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:50.148190    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-242ddaf0-7b7b974bc-5rsm9" event={"ID":"706bd83a-953d-4b2e-8f1c-e51bf4343a19","Type":"ContainerDied","Data":"36c76f9df41456e4bae3c6301c96c882b80f8a2b00cf7b961c3a9e21ecb05597"}
Oct 30 22:18:50.157817 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:50.157600    4473 scope.go:122] "RemoveContainer" containerID="2aa491f0eb496338cbc817794a6e52e9bc7227a25da76367bff6045f176d81bb"
Oct 30 22:18:50.158033 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:18:50.158012    4473 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2aa491f0eb496338cbc817794a6e52e9bc7227a25da76367bff6045f176d81bb\": container with ID starting with 2aa491f0eb496338cbc817794a6e52e9bc7227a25da76367bff6045f176d81bb not found: ID does not exist" containerID="2aa491f0eb496338cbc817794a6e52e9bc7227a25da76367bff6045f176d81bb"
Oct 30 22:18:50.158084 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:50.158042    4473 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2aa491f0eb496338cbc817794a6e52e9bc7227a25da76367bff6045f176d81bb"} err="failed to get container status \"2aa491f0eb496338cbc817794a6e52e9bc7227a25da76367bff6045f176d81bb\": rpc error: code = NotFound desc = could not find container \"2aa491f0eb496338cbc817794a6e52e9bc7227a25da76367bff6045f176d81bb\": container with ID starting with 2aa491f0eb496338cbc817794a6e52e9bc7227a25da76367bff6045f176d81bb not found: ID does not exist"
Oct 30 22:18:50.158084 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:50.158064    4473 scope.go:122] "RemoveContainer" containerID="79868169be9b695577a671c7b4eac2561bce79cfa7f25fbd94d5c89f4bb87cb0"
Oct 30 22:18:50.160948 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:50.160894    4473 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/706bd83a-953d-4b2e-8f1c-e51bf4343a19-plugins-config-volume\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:18:50.160948 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:50.160943    4473 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-czbtn\" (UniqueName: \"kubernetes.io/projected/706bd83a-953d-4b2e-8f1c-e51bf4343a19-kube-api-access-czbtn\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:18:50.168430 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:50.168408    4473 scope.go:122] "RemoveContainer" containerID="79868169be9b695577a671c7b4eac2561bce79cfa7f25fbd94d5c89f4bb87cb0"
Oct 30 22:18:50.168748 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:18:50.168722    4473 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"79868169be9b695577a671c7b4eac2561bce79cfa7f25fbd94d5c89f4bb87cb0\": container with ID starting with 79868169be9b695577a671c7b4eac2561bce79cfa7f25fbd94d5c89f4bb87cb0 not found: ID does not exist" containerID="79868169be9b695577a671c7b4eac2561bce79cfa7f25fbd94d5c89f4bb87cb0"
Oct 30 22:18:50.168852 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:50.168755    4473 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"79868169be9b695577a671c7b4eac2561bce79cfa7f25fbd94d5c89f4bb87cb0"} err="failed to get container status \"79868169be9b695577a671c7b4eac2561bce79cfa7f25fbd94d5c89f4bb87cb0\": rpc error: code = NotFound desc = could not find container \"79868169be9b695577a671c7b4eac2561bce79cfa7f25fbd94d5c89f4bb87cb0\": container with ID starting with 79868169be9b695577a671c7b4eac2561bce79cfa7f25fbd94d5c89f4bb87cb0 not found: ID does not exist"
Oct 30 22:18:50.315048 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:50.314884    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-242ddaf0-576ccc4d89-28sq2"]
Oct 30 22:18:50.317163 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:50.317137    4473 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-242ddaf0-576ccc4d89-28sq2"]
Oct 30 22:18:50.329170 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:18:50.329129    4473 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fncvg" containerName="extractor"
Oct 30 22:18:50.356087 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:50.356054    4473 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7665007f-2e07-47f7-ac42-82dfd53d85a1" path="/var/lib/kubelet/pods/7665007f-2e07-47f7-ac42-82dfd53d85a1/volumes"
Oct 30 22:18:50.356572 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:50.356552    4473 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="cd728188-2246-4f1d-8f18-ec351a96b935" path="/var/lib/kubelet/pods/cd728188-2246-4f1d-8f18-ec351a96b935/volumes"
Oct 30 22:18:50.428068 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:50.428029    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-242ddaf0-7b7b974bc-5rsm9"]
Oct 30 22:18:50.440709 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:50.433809    4473 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-242ddaf0-7b7b974bc-5rsm9"]
Oct 30 22:18:52.356413 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:52.356378    4473 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="706bd83a-953d-4b2e-8f1c-e51bf4343a19" path="/var/lib/kubelet/pods/706bd83a-953d-4b2e-8f1c-e51bf4343a19/volumes"
Oct 30 22:18:54.741206 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:54.741169    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-dde1e8-7d47cf4786-8d75q"]
Oct 30 22:18:54.741614 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:54.741378    4473 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-dde1e8-7d47cf4786-8d75q" podUID="b372cc09-4720-4855-9c3a-0e8e7b8beb70" containerName="maas-api" containerID="cri-o://d37b59a9ced77e902cd29542a330609e06d4e6055e5fa7e9b482356aa3fdf4e0" gracePeriod=30
Oct 30 22:18:55.032891 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:55.032860    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-dde1e8-7d47cf4786-8d75q"
Oct 30 22:18:55.118829 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:55.118794    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b372cc09-4720-4855-9c3a-0e8e7b8beb70-metrics-certs\" (UniqueName: \"kubernetes.io/secret/b372cc09-4720-4855-9c3a-0e8e7b8beb70-metrics-certs\") pod \"b372cc09-4720-4855-9c3a-0e8e7b8beb70\" (UID: \"b372cc09-4720-4855-9c3a-0e8e7b8beb70\") "
Oct 30 22:18:55.119024 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:55.118889    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b372cc09-4720-4855-9c3a-0e8e7b8beb70-kube-api-access-bz2lh\" (UniqueName: \"kubernetes.io/projected/b372cc09-4720-4855-9c3a-0e8e7b8beb70-kube-api-access-bz2lh\") pod \"b372cc09-4720-4855-9c3a-0e8e7b8beb70\" (UID: \"b372cc09-4720-4855-9c3a-0e8e7b8beb70\") "
Oct 30 22:18:55.119024 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:55.119006    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b372cc09-4720-4855-9c3a-0e8e7b8beb70-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b372cc09-4720-4855-9c3a-0e8e7b8beb70-maas-api-tls\") pod \"b372cc09-4720-4855-9c3a-0e8e7b8beb70\" (UID: \"b372cc09-4720-4855-9c3a-0e8e7b8beb70\") "
Oct 30 22:18:55.119138 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:55.119027    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b372cc09-4720-4855-9c3a-0e8e7b8beb70-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b372cc09-4720-4855-9c3a-0e8e7b8beb70-openshift-service-ca\") pod \"b372cc09-4720-4855-9c3a-0e8e7b8beb70\" (UID: \"b372cc09-4720-4855-9c3a-0e8e7b8beb70\") "
Oct 30 22:18:55.119597 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:55.119565    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b372cc09-4720-4855-9c3a-0e8e7b8beb70-openshift-service-ca" pod "b372cc09-4720-4855-9c3a-0e8e7b8beb70" (UID: "b372cc09-4720-4855-9c3a-0e8e7b8beb70"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:18:55.121204 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:55.121183    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b372cc09-4720-4855-9c3a-0e8e7b8beb70-kube-api-access-bz2lh" pod "b372cc09-4720-4855-9c3a-0e8e7b8beb70" (UID: "b372cc09-4720-4855-9c3a-0e8e7b8beb70"). InnerVolumeSpecName "kube-api-access-bz2lh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:18:55.121307 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:55.121279    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b372cc09-4720-4855-9c3a-0e8e7b8beb70-metrics-certs" pod "b372cc09-4720-4855-9c3a-0e8e7b8beb70" (UID: "b372cc09-4720-4855-9c3a-0e8e7b8beb70"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:18:55.121364 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:55.121307    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b372cc09-4720-4855-9c3a-0e8e7b8beb70-maas-api-tls" pod "b372cc09-4720-4855-9c3a-0e8e7b8beb70" (UID: "b372cc09-4720-4855-9c3a-0e8e7b8beb70"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:18:55.181296 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:55.181256    4473 generic.go:356] "Generic (PLEG): container finished" podID="b372cc09-4720-4855-9c3a-0e8e7b8beb70" containerID="d37b59a9ced77e902cd29542a330609e06d4e6055e5fa7e9b482356aa3fdf4e0" exitCode=0
Oct 30 22:18:55.181470 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:55.181302    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-dde1e8-7d47cf4786-8d75q"
Oct 30 22:18:55.181470 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:55.181339    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-dde1e8-7d47cf4786-8d75q" event={"ID":"b372cc09-4720-4855-9c3a-0e8e7b8beb70","Type":"ContainerDied","Data":"d37b59a9ced77e902cd29542a330609e06d4e6055e5fa7e9b482356aa3fdf4e0"}
Oct 30 22:18:55.181470 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:55.181395    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-dde1e8-7d47cf4786-8d75q" event={"ID":"b372cc09-4720-4855-9c3a-0e8e7b8beb70","Type":"ContainerDied","Data":"ada95f347f28865d75ad9c6b62f7f1ce7bf5e78eef56b25ac7c430db937ed45d"}
Oct 30 22:18:55.181470 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:55.181423    4473 scope.go:122] "RemoveContainer" containerID="d37b59a9ced77e902cd29542a330609e06d4e6055e5fa7e9b482356aa3fdf4e0"
Oct 30 22:18:55.196619 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:55.196596    4473 scope.go:122] "RemoveContainer" containerID="d37b59a9ced77e902cd29542a330609e06d4e6055e5fa7e9b482356aa3fdf4e0"
Oct 30 22:18:55.196978 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:18:55.196958    4473 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d37b59a9ced77e902cd29542a330609e06d4e6055e5fa7e9b482356aa3fdf4e0\": container with ID starting with d37b59a9ced77e902cd29542a330609e06d4e6055e5fa7e9b482356aa3fdf4e0 not found: ID does not exist" containerID="d37b59a9ced77e902cd29542a330609e06d4e6055e5fa7e9b482356aa3fdf4e0"
Oct 30 22:18:55.197036 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:55.196989    4473 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d37b59a9ced77e902cd29542a330609e06d4e6055e5fa7e9b482356aa3fdf4e0"} err="failed to get container status \"d37b59a9ced77e902cd29542a330609e06d4e6055e5fa7e9b482356aa3fdf4e0\": rpc error: code = NotFound desc = could not find container \"d37b59a9ced77e902cd29542a330609e06d4e6055e5fa7e9b482356aa3fdf4e0\": container with ID starting with d37b59a9ced77e902cd29542a330609e06d4e6055e5fa7e9b482356aa3fdf4e0 not found: ID does not exist"
Oct 30 22:18:55.220811 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:55.220784    4473 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b372cc09-4720-4855-9c3a-0e8e7b8beb70-maas-api-tls\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:18:55.220811 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:55.220810    4473 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b372cc09-4720-4855-9c3a-0e8e7b8beb70-openshift-service-ca\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:18:55.220811 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:55.220820    4473 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b372cc09-4720-4855-9c3a-0e8e7b8beb70-metrics-certs\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:18:55.221060 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:55.220829    4473 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bz2lh\" (UniqueName: \"kubernetes.io/projected/b372cc09-4720-4855-9c3a-0e8e7b8beb70-kube-api-access-bz2lh\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:18:55.353693 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:55.353613    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-dde1e8-7d47cf4786-8d75q"]
Oct 30 22:18:55.357485 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:55.357460    4473 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-dde1e8-7d47cf4786-8d75q"]
Oct 30 22:18:56.359674 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:18:56.359636    4473 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b372cc09-4720-4855-9c3a-0e8e7b8beb70" path="/var/lib/kubelet/pods/b372cc09-4720-4855-9c3a-0e8e7b8beb70/volumes"
Oct 30 22:19:02.727745 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:19:02.727708    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-242ddaf0-5d8df4c987-7lkww"]
Oct 30 22:19:02.728229 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:19:02.727887    4473 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-242ddaf0-5d8df4c987-7lkww" podUID="0dcc6ba9-67e6-4ec3-9315-305edff7967e" containerName="maas-api" containerID="cri-o://5534f02179bc9652216b3fda8156c7573f5bc0d6df7ade7b669d41a834ae41e3" gracePeriod=30
Oct 30 22:19:03.021760 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:19:03.021735    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-242ddaf0-5d8df4c987-7lkww"
Oct 30 22:19:03.130296 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:19:03.130251    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0dcc6ba9-67e6-4ec3-9315-305edff7967e-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0dcc6ba9-67e6-4ec3-9315-305edff7967e-maas-api-tls\") pod \"0dcc6ba9-67e6-4ec3-9315-305edff7967e\" (UID: \"0dcc6ba9-67e6-4ec3-9315-305edff7967e\") "
Oct 30 22:19:03.130511 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:19:03.130338    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0dcc6ba9-67e6-4ec3-9315-305edff7967e-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0dcc6ba9-67e6-4ec3-9315-305edff7967e-openshift-service-ca\") pod \"0dcc6ba9-67e6-4ec3-9315-305edff7967e\" (UID: \"0dcc6ba9-67e6-4ec3-9315-305edff7967e\") "
Oct 30 22:19:03.130511 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:19:03.130378    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0dcc6ba9-67e6-4ec3-9315-305edff7967e-metrics-certs\" (UniqueName: \"kubernetes.io/secret/0dcc6ba9-67e6-4ec3-9315-305edff7967e-metrics-certs\") pod \"0dcc6ba9-67e6-4ec3-9315-305edff7967e\" (UID: \"0dcc6ba9-67e6-4ec3-9315-305edff7967e\") "
Oct 30 22:19:03.130511 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:19:03.130484    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0dcc6ba9-67e6-4ec3-9315-305edff7967e-kube-api-access-f66vg\" (UniqueName: \"kubernetes.io/projected/0dcc6ba9-67e6-4ec3-9315-305edff7967e-kube-api-access-f66vg\") pod \"0dcc6ba9-67e6-4ec3-9315-305edff7967e\" (UID: \"0dcc6ba9-67e6-4ec3-9315-305edff7967e\") "
Oct 30 22:19:03.131574 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:19:03.131278    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0dcc6ba9-67e6-4ec3-9315-305edff7967e-openshift-service-ca" pod "0dcc6ba9-67e6-4ec3-9315-305edff7967e" (UID: "0dcc6ba9-67e6-4ec3-9315-305edff7967e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:19:03.147024 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:19:03.146496    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0dcc6ba9-67e6-4ec3-9315-305edff7967e-maas-api-tls" pod "0dcc6ba9-67e6-4ec3-9315-305edff7967e" (UID: "0dcc6ba9-67e6-4ec3-9315-305edff7967e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:19:03.147024 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:19:03.146885    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0dcc6ba9-67e6-4ec3-9315-305edff7967e-kube-api-access-f66vg" pod "0dcc6ba9-67e6-4ec3-9315-305edff7967e" (UID: "0dcc6ba9-67e6-4ec3-9315-305edff7967e"). InnerVolumeSpecName "kube-api-access-f66vg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:19:03.148343 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:19:03.148307    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0dcc6ba9-67e6-4ec3-9315-305edff7967e-metrics-certs" pod "0dcc6ba9-67e6-4ec3-9315-305edff7967e" (UID: "0dcc6ba9-67e6-4ec3-9315-305edff7967e"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:19:03.230797 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:19:03.230766    4473 generic.go:356] "Generic (PLEG): container finished" podID="0dcc6ba9-67e6-4ec3-9315-305edff7967e" containerID="5534f02179bc9652216b3fda8156c7573f5bc0d6df7ade7b669d41a834ae41e3" exitCode=0
Oct 30 22:19:03.230992 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:19:03.230810    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-242ddaf0-5d8df4c987-7lkww"
Oct 30 22:19:03.230992 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:19:03.230851    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-242ddaf0-5d8df4c987-7lkww" event={"ID":"0dcc6ba9-67e6-4ec3-9315-305edff7967e","Type":"ContainerDied","Data":"5534f02179bc9652216b3fda8156c7573f5bc0d6df7ade7b669d41a834ae41e3"}
Oct 30 22:19:03.230992 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:19:03.230909    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-242ddaf0-5d8df4c987-7lkww" event={"ID":"0dcc6ba9-67e6-4ec3-9315-305edff7967e","Type":"ContainerDied","Data":"1c7d467b145820a421b869e9ed19a20e67defba51688a1456b2afa87685ca3f0"}
Oct 30 22:19:03.230992 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:19:03.230938    4473 scope.go:122] "RemoveContainer" containerID="5534f02179bc9652216b3fda8156c7573f5bc0d6df7ade7b669d41a834ae41e3"
Oct 30 22:19:03.231526 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:19:03.231503    4473 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0dcc6ba9-67e6-4ec3-9315-305edff7967e-openshift-service-ca\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:19:03.231526 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:19:03.231525    4473 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0dcc6ba9-67e6-4ec3-9315-305edff7967e-metrics-certs\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:19:03.231737 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:19:03.231536    4473 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f66vg\" (UniqueName: \"kubernetes.io/projected/0dcc6ba9-67e6-4ec3-9315-305edff7967e-kube-api-access-f66vg\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:19:03.231737 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:19:03.231546    4473 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0dcc6ba9-67e6-4ec3-9315-305edff7967e-maas-api-tls\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:19:03.241459 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:19:03.241436    4473 scope.go:122] "RemoveContainer" containerID="5534f02179bc9652216b3fda8156c7573f5bc0d6df7ade7b669d41a834ae41e3"
Oct 30 22:19:03.241730 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:19:03.241712    4473 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5534f02179bc9652216b3fda8156c7573f5bc0d6df7ade7b669d41a834ae41e3\": container with ID starting with 5534f02179bc9652216b3fda8156c7573f5bc0d6df7ade7b669d41a834ae41e3 not found: ID does not exist" containerID="5534f02179bc9652216b3fda8156c7573f5bc0d6df7ade7b669d41a834ae41e3"
Oct 30 22:19:03.241783 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:19:03.241740    4473 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5534f02179bc9652216b3fda8156c7573f5bc0d6df7ade7b669d41a834ae41e3"} err="failed to get container status \"5534f02179bc9652216b3fda8156c7573f5bc0d6df7ade7b669d41a834ae41e3\": rpc error: code = NotFound desc = could not find container \"5534f02179bc9652216b3fda8156c7573f5bc0d6df7ade7b669d41a834ae41e3\": container with ID starting with 5534f02179bc9652216b3fda8156c7573f5bc0d6df7ade7b669d41a834ae41e3 not found: ID does not exist"
Oct 30 22:19:03.396565 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:19:03.396527    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-242ddaf0-5d8df4c987-7lkww"]
Oct 30 22:19:03.400895 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:19:03.400866    4473 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-242ddaf0-5d8df4c987-7lkww"]
Oct 30 22:19:03.976907 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:19:03.976869    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-fnz9z"]
Oct 30 22:19:03.977523 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:19:03.977136    4473 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-fnz9z" podUID="f46a30dc-0abe-4906-8b58-c439a0bff711" containerName="istio-proxy" containerID="cri-o://deae3cac1ce1326b33a78cf69d9fe0fb567fad5a2944340b2c89679e7406e439" gracePeriod=30
Oct 30 22:19:04.355910 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:19:04.355833    4473 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0dcc6ba9-67e6-4ec3-9315-305edff7967e" path="/var/lib/kubelet/pods/0dcc6ba9-67e6-4ec3-9315-305edff7967e/volumes"
Oct 30 22:19:09.233113 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:19:09.233084    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-fnz9z"
Oct 30 22:19:09.265138 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:19:09.265050    4473 generic.go:356] "Generic (PLEG): container finished" podID="f46a30dc-0abe-4906-8b58-c439a0bff711" containerID="deae3cac1ce1326b33a78cf69d9fe0fb567fad5a2944340b2c89679e7406e439" exitCode=0
Oct 30 22:19:09.265138 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:19:09.265103    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-fnz9z"
Oct 30 22:19:09.265333 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:19:09.265142    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-fnz9z" event={"ID":"f46a30dc-0abe-4906-8b58-c439a0bff711","Type":"ContainerDied","Data":"deae3cac1ce1326b33a78cf69d9fe0fb567fad5a2944340b2c89679e7406e439"}
Oct 30 22:19:09.265333 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:19:09.265203    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-fnz9z" event={"ID":"f46a30dc-0abe-4906-8b58-c439a0bff711","Type":"ContainerDied","Data":"afce6ad9496157a686c547656e42403757a3f7b8094a92fb5132aca01377af1a"}
Oct 30 22:19:09.265333 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:19:09.265229    4473 scope.go:122] "RemoveContainer" containerID="deae3cac1ce1326b33a78cf69d9fe0fb567fad5a2944340b2c89679e7406e439"
Oct 30 22:19:09.274751 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:19:09.274729    4473 scope.go:122] "RemoveContainer" containerID="deae3cac1ce1326b33a78cf69d9fe0fb567fad5a2944340b2c89679e7406e439"
Oct 30 22:19:09.275123 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:19:09.275102    4473 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"deae3cac1ce1326b33a78cf69d9fe0fb567fad5a2944340b2c89679e7406e439\": container with ID starting with deae3cac1ce1326b33a78cf69d9fe0fb567fad5a2944340b2c89679e7406e439 not found: ID does not exist" containerID="deae3cac1ce1326b33a78cf69d9fe0fb567fad5a2944340b2c89679e7406e439"
Oct 30 22:19:09.275185 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:19:09.275135    4473 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"deae3cac1ce1326b33a78cf69d9fe0fb567fad5a2944340b2c89679e7406e439"} err="failed to get container status \"deae3cac1ce1326b33a78cf69d9fe0fb567fad5a2944340b2c89679e7406e439\": rpc error: code = NotFound desc = could not find container \"deae3cac1ce1326b33a78cf69d9fe0fb567fad5a2944340b2c89679e7406e439\": container with ID starting with deae3cac1ce1326b33a78cf69d9fe0fb567fad5a2944340b2c89679e7406e439 not found: ID does not exist"
Oct 30 22:19:09.399244 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:19:09.399210    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f46a30dc-0abe-4906-8b58-c439a0bff711-kube-api-access-4ktvc\" (UniqueName: \"kubernetes.io/projected/f46a30dc-0abe-4906-8b58-c439a0bff711-kube-api-access-4ktvc\") pod \"f46a30dc-0abe-4906-8b58-c439a0bff711\" (UID: \"f46a30dc-0abe-4906-8b58-c439a0bff711\") "
Oct 30 22:19:09.399477 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:19:09.399264    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f46a30dc-0abe-4906-8b58-c439a0bff711-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f46a30dc-0abe-4906-8b58-c439a0bff711-istio-data\") pod \"f46a30dc-0abe-4906-8b58-c439a0bff711\" (UID: \"f46a30dc-0abe-4906-8b58-c439a0bff711\") "
Oct 30 22:19:09.399477 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:19:09.399299    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f46a30dc-0abe-4906-8b58-c439a0bff711-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f46a30dc-0abe-4906-8b58-c439a0bff711-istio-envoy\") pod \"f46a30dc-0abe-4906-8b58-c439a0bff711\" (UID: \"f46a30dc-0abe-4906-8b58-c439a0bff711\") "
Oct 30 22:19:09.399477 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:19:09.399390    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f46a30dc-0abe-4906-8b58-c439a0bff711-istio-token\" (UniqueName: \"kubernetes.io/projected/f46a30dc-0abe-4906-8b58-c439a0bff711-istio-token\") pod \"f46a30dc-0abe-4906-8b58-c439a0bff711\" (UID: \"f46a30dc-0abe-4906-8b58-c439a0bff711\") "
Oct 30 22:19:09.399477 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:19:09.399461    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f46a30dc-0abe-4906-8b58-c439a0bff711-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f46a30dc-0abe-4906-8b58-c439a0bff711-workload-socket\") pod \"f46a30dc-0abe-4906-8b58-c439a0bff711\" (UID: \"f46a30dc-0abe-4906-8b58-c439a0bff711\") "
Oct 30 22:19:09.399715 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:19:09.399521    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f46a30dc-0abe-4906-8b58-c439a0bff711-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f46a30dc-0abe-4906-8b58-c439a0bff711-istiod-ca-cert\") pod \"f46a30dc-0abe-4906-8b58-c439a0bff711\" (UID: \"f46a30dc-0abe-4906-8b58-c439a0bff711\") "
Oct 30 22:19:09.399715 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:19:09.399596    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f46a30dc-0abe-4906-8b58-c439a0bff711-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f46a30dc-0abe-4906-8b58-c439a0bff711-credential-socket\") pod \"f46a30dc-0abe-4906-8b58-c439a0bff711\" (UID: \"f46a30dc-0abe-4906-8b58-c439a0bff711\") "
Oct 30 22:19:09.399715 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:19:09.399634    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f46a30dc-0abe-4906-8b58-c439a0bff711-istio-data" pod "f46a30dc-0abe-4906-8b58-c439a0bff711" (UID: "f46a30dc-0abe-4906-8b58-c439a0bff711"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:19:09.399715 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:19:09.399675    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f46a30dc-0abe-4906-8b58-c439a0bff711-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f46a30dc-0abe-4906-8b58-c439a0bff711-workload-certs\") pod \"f46a30dc-0abe-4906-8b58-c439a0bff711\" (UID: \"f46a30dc-0abe-4906-8b58-c439a0bff711\") "
Oct 30 22:19:09.399913 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:19:09.399728    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/f46a30dc-0abe-4906-8b58-c439a0bff711-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f46a30dc-0abe-4906-8b58-c439a0bff711-istio-podinfo\") pod \"f46a30dc-0abe-4906-8b58-c439a0bff711\" (UID: \"f46a30dc-0abe-4906-8b58-c439a0bff711\") "
Oct 30 22:19:09.399913 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:19:09.399729    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f46a30dc-0abe-4906-8b58-c439a0bff711-workload-socket" pod "f46a30dc-0abe-4906-8b58-c439a0bff711" (UID: "f46a30dc-0abe-4906-8b58-c439a0bff711"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:19:09.399913 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:19:09.399894    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f46a30dc-0abe-4906-8b58-c439a0bff711-credential-socket" pod "f46a30dc-0abe-4906-8b58-c439a0bff711" (UID: "f46a30dc-0abe-4906-8b58-c439a0bff711"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:19:09.400109 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:19:09.399959    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f46a30dc-0abe-4906-8b58-c439a0bff711-istiod-ca-cert" pod "f46a30dc-0abe-4906-8b58-c439a0bff711" (UID: "f46a30dc-0abe-4906-8b58-c439a0bff711"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:19:09.400109 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:19:09.400008    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f46a30dc-0abe-4906-8b58-c439a0bff711-workload-certs" pod "f46a30dc-0abe-4906-8b58-c439a0bff711" (UID: "f46a30dc-0abe-4906-8b58-c439a0bff711"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:19:09.400324 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:19:09.400305    4473 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f46a30dc-0abe-4906-8b58-c439a0bff711-workload-socket\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:19:09.400414 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:19:09.400329    4473 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f46a30dc-0abe-4906-8b58-c439a0bff711-istiod-ca-cert\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:19:09.400414 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:19:09.400343    4473 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f46a30dc-0abe-4906-8b58-c439a0bff711-credential-socket\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:19:09.400414 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:19:09.400356    4473 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f46a30dc-0abe-4906-8b58-c439a0bff711-workload-certs\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:19:09.400414 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:19:09.400395    4473 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f46a30dc-0abe-4906-8b58-c439a0bff711-istio-data\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:19:09.402071 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:19:09.402042    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f46a30dc-0abe-4906-8b58-c439a0bff711-istio-podinfo" pod "f46a30dc-0abe-4906-8b58-c439a0bff711" (UID: "f46a30dc-0abe-4906-8b58-c439a0bff711"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:19:09.402071 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:19:09.402059    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f46a30dc-0abe-4906-8b58-c439a0bff711-kube-api-access-4ktvc" pod "f46a30dc-0abe-4906-8b58-c439a0bff711" (UID: "f46a30dc-0abe-4906-8b58-c439a0bff711"). InnerVolumeSpecName "kube-api-access-4ktvc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:19:09.402269 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:19:09.402250    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f46a30dc-0abe-4906-8b58-c439a0bff711-istio-token" pod "f46a30dc-0abe-4906-8b58-c439a0bff711" (UID: "f46a30dc-0abe-4906-8b58-c439a0bff711"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:19:09.402326 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:19:09.402310    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f46a30dc-0abe-4906-8b58-c439a0bff711-istio-envoy" pod "f46a30dc-0abe-4906-8b58-c439a0bff711" (UID: "f46a30dc-0abe-4906-8b58-c439a0bff711"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:19:09.501181 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:19:09.501135    4473 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f46a30dc-0abe-4906-8b58-c439a0bff711-istio-podinfo\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:19:09.501181 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:19:09.501167    4473 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4ktvc\" (UniqueName: \"kubernetes.io/projected/f46a30dc-0abe-4906-8b58-c439a0bff711-kube-api-access-4ktvc\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:19:09.501181 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:19:09.501178    4473 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f46a30dc-0abe-4906-8b58-c439a0bff711-istio-envoy\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:19:09.501181 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:19:09.501186    4473 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f46a30dc-0abe-4906-8b58-c439a0bff711-istio-token\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:19:09.687953 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:19:09.687900    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-fnz9z"]
Oct 30 22:19:09.692978 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:19:09.692947    4473 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-dde1e8-openshift-default-7c667f55ff-fnz9z"]
Oct 30 22:19:10.264303 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:19:10.264265    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-242ddaf0/e2e-lifecycle-model-242ddaf0-kserve-87686648f-l9tjf"]
Oct 30 22:19:10.358535 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:19:10.358500    4473 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f46a30dc-0abe-4906-8b58-c439a0bff711" path="/var/lib/kubelet/pods/f46a30dc-0abe-4906-8b58-c439a0bff711/volumes"
Oct 30 22:19:20.329149 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:19:20.329051    4473 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fncvg" containerName="extractor"
Oct 30 22:19:25.286494 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:19:25.286445    4473 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-mt-242ddaf0/e2e-lifecycle-model-242ddaf0-kserve-87686648f-l9tjf" podUID="5fb7086a-96b1-4b1c-a86d-8e38bca4cf80" containerName="main" containerID="cri-o://88390cd46f489f97c53e8647fbbaf86c6d03b7585c2053bf8237d103a3968a5f" gracePeriod=45
Oct 30 22:19:25.550241 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:19:25.550176    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-242ddaf0/e2e-lifecycle-model-242ddaf0-kserve-87686648f-l9tjf"
Oct 30 22:19:25.572351 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:19:25.572320    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5fb7086a-96b1-4b1c-a86d-8e38bca4cf80-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5fb7086a-96b1-4b1c-a86d-8e38bca4cf80-model-cache\") pod \"5fb7086a-96b1-4b1c-a86d-8e38bca4cf80\" (UID: \"5fb7086a-96b1-4b1c-a86d-8e38bca4cf80\") "
Oct 30 22:19:25.572540 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:19:25.572370    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5fb7086a-96b1-4b1c-a86d-8e38bca4cf80-dshm\" (UniqueName: \"kubernetes.io/empty-dir/5fb7086a-96b1-4b1c-a86d-8e38bca4cf80-dshm\") pod \"5fb7086a-96b1-4b1c-a86d-8e38bca4cf80\" (UID: \"5fb7086a-96b1-4b1c-a86d-8e38bca4cf80\") "
Oct 30 22:19:25.572540 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:19:25.572389    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5fb7086a-96b1-4b1c-a86d-8e38bca4cf80-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5fb7086a-96b1-4b1c-a86d-8e38bca4cf80-tmp-dir\") pod \"5fb7086a-96b1-4b1c-a86d-8e38bca4cf80\" (UID: \"5fb7086a-96b1-4b1c-a86d-8e38bca4cf80\") "
Oct 30 22:19:25.572540 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:19:25.572428    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5fb7086a-96b1-4b1c-a86d-8e38bca4cf80-kube-api-access-d6rht\" (UniqueName: \"kubernetes.io/projected/5fb7086a-96b1-4b1c-a86d-8e38bca4cf80-kube-api-access-d6rht\") pod \"5fb7086a-96b1-4b1c-a86d-8e38bca4cf80\" (UID: \"5fb7086a-96b1-4b1c-a86d-8e38bca4cf80\") "
Oct 30 22:19:25.572540 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:19:25.572539    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5fb7086a-96b1-4b1c-a86d-8e38bca4cf80-tls-certs\" (UniqueName: \"kubernetes.io/secret/5fb7086a-96b1-4b1c-a86d-8e38bca4cf80-tls-certs\") pod \"5fb7086a-96b1-4b1c-a86d-8e38bca4cf80\" (UID: \"5fb7086a-96b1-4b1c-a86d-8e38bca4cf80\") "
Oct 30 22:19:25.572774 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:19:25.572600    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5fb7086a-96b1-4b1c-a86d-8e38bca4cf80-home\" (UniqueName: \"kubernetes.io/empty-dir/5fb7086a-96b1-4b1c-a86d-8e38bca4cf80-home\") pod \"5fb7086a-96b1-4b1c-a86d-8e38bca4cf80\" (UID: \"5fb7086a-96b1-4b1c-a86d-8e38bca4cf80\") "
Oct 30 22:19:25.572774 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:19:25.572658    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5fb7086a-96b1-4b1c-a86d-8e38bca4cf80-model-cache" pod "5fb7086a-96b1-4b1c-a86d-8e38bca4cf80" (UID: "5fb7086a-96b1-4b1c-a86d-8e38bca4cf80"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:19:25.572774 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:19:25.572726    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5fb7086a-96b1-4b1c-a86d-8e38bca4cf80-tmp-dir" pod "5fb7086a-96b1-4b1c-a86d-8e38bca4cf80" (UID: "5fb7086a-96b1-4b1c-a86d-8e38bca4cf80"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:19:25.573028 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:19:25.573002    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5fb7086a-96b1-4b1c-a86d-8e38bca4cf80-home" pod "5fb7086a-96b1-4b1c-a86d-8e38bca4cf80" (UID: "5fb7086a-96b1-4b1c-a86d-8e38bca4cf80"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:19:25.573111 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:19:25.573073    4473 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5fb7086a-96b1-4b1c-a86d-8e38bca4cf80-model-cache\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:19:25.573111 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:19:25.573094    4473 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5fb7086a-96b1-4b1c-a86d-8e38bca4cf80-tmp-dir\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:19:25.574954 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:19:25.574897    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5fb7086a-96b1-4b1c-a86d-8e38bca4cf80-dshm" pod "5fb7086a-96b1-4b1c-a86d-8e38bca4cf80" (UID: "5fb7086a-96b1-4b1c-a86d-8e38bca4cf80"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:19:25.575054 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:19:25.574981    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5fb7086a-96b1-4b1c-a86d-8e38bca4cf80-tls-certs" pod "5fb7086a-96b1-4b1c-a86d-8e38bca4cf80" (UID: "5fb7086a-96b1-4b1c-a86d-8e38bca4cf80"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:19:25.575054 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:19:25.575012    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5fb7086a-96b1-4b1c-a86d-8e38bca4cf80-kube-api-access-d6rht" pod "5fb7086a-96b1-4b1c-a86d-8e38bca4cf80" (UID: "5fb7086a-96b1-4b1c-a86d-8e38bca4cf80"). InnerVolumeSpecName "kube-api-access-d6rht". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:19:25.677186 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:19:25.677157    4473 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5fb7086a-96b1-4b1c-a86d-8e38bca4cf80-dshm\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:19:25.677186 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:19:25.677188    4473 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d6rht\" (UniqueName: \"kubernetes.io/projected/5fb7086a-96b1-4b1c-a86d-8e38bca4cf80-kube-api-access-d6rht\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:19:25.677414 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:19:25.677204    4473 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5fb7086a-96b1-4b1c-a86d-8e38bca4cf80-tls-certs\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:19:25.677414 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:19:25.677219    4473 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5fb7086a-96b1-4b1c-a86d-8e38bca4cf80-home\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:19:26.364518 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:19:26.364480    4473 generic.go:356] "Generic (PLEG): container finished" podID="5fb7086a-96b1-4b1c-a86d-8e38bca4cf80" containerID="88390cd46f489f97c53e8647fbbaf86c6d03b7585c2053bf8237d103a3968a5f" exitCode=0
Oct 30 22:19:26.364979 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:19:26.364534    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-242ddaf0/e2e-lifecycle-model-242ddaf0-kserve-87686648f-l9tjf"
Oct 30 22:19:26.364979 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:19:26.364571    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-242ddaf0/e2e-lifecycle-model-242ddaf0-kserve-87686648f-l9tjf" event={"ID":"5fb7086a-96b1-4b1c-a86d-8e38bca4cf80","Type":"ContainerDied","Data":"88390cd46f489f97c53e8647fbbaf86c6d03b7585c2053bf8237d103a3968a5f"}
Oct 30 22:19:26.364979 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:19:26.364617    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-242ddaf0/e2e-lifecycle-model-242ddaf0-kserve-87686648f-l9tjf" event={"ID":"5fb7086a-96b1-4b1c-a86d-8e38bca4cf80","Type":"ContainerDied","Data":"c90baa8711af800734967126be53bb571c78e4be042fb3eaa7f5e3ec9cf6e5d0"}
Oct 30 22:19:26.364979 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:19:26.364635    4473 scope.go:122] "RemoveContainer" containerID="88390cd46f489f97c53e8647fbbaf86c6d03b7585c2053bf8237d103a3968a5f"
Oct 30 22:19:26.375141 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:19:26.375118    4473 scope.go:122] "RemoveContainer" containerID="88390cd46f489f97c53e8647fbbaf86c6d03b7585c2053bf8237d103a3968a5f"
Oct 30 22:19:26.375452 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:19:26.375430    4473 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"88390cd46f489f97c53e8647fbbaf86c6d03b7585c2053bf8237d103a3968a5f\": container with ID starting with 88390cd46f489f97c53e8647fbbaf86c6d03b7585c2053bf8237d103a3968a5f not found: ID does not exist" containerID="88390cd46f489f97c53e8647fbbaf86c6d03b7585c2053bf8237d103a3968a5f"
Oct 30 22:19:26.375522 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:19:26.375461    4473 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"88390cd46f489f97c53e8647fbbaf86c6d03b7585c2053bf8237d103a3968a5f"} err="failed to get container status \"88390cd46f489f97c53e8647fbbaf86c6d03b7585c2053bf8237d103a3968a5f\": rpc error: code = NotFound desc = could not find container \"88390cd46f489f97c53e8647fbbaf86c6d03b7585c2053bf8237d103a3968a5f\": container with ID starting with 88390cd46f489f97c53e8647fbbaf86c6d03b7585c2053bf8237d103a3968a5f not found: ID does not exist"
Oct 30 22:19:26.530352 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:19:26.530318    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-242ddaf0/e2e-lifecycle-model-242ddaf0-kserve-87686648f-l9tjf"]
Oct 30 22:19:26.580666 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:19:26.580627    4473 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-242ddaf0/e2e-lifecycle-model-242ddaf0-kserve-87686648f-l9tjf"]
Oct 30 22:19:28.356064 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:19:28.356033    4473 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5fb7086a-96b1-4b1c-a86d-8e38bca4cf80" path="/var/lib/kubelet/pods/5fb7086a-96b1-4b1c-a86d-8e38bca4cf80/volumes"
Oct 30 22:19:50.324779 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:19:50.324732    4473 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fncvg" containerName="extractor"
Oct 30 22:19:51.021440 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:19:51.021400    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w6-29cbad/premium-w6-29cbad-kserve-6c9f8d6899-qtv6c"]
Oct 30 22:19:58.673902 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:19:58.673858    4473 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-6826e712-c97kp"]
Oct 30 22:19:58.674604 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:19:58.674582    4473 cpu_manager.go:414] "containerMap: removing container" podUID="5fb7086a-96b1-4b1c-a86d-8e38bca4cf80" containerName="main"
Oct 30 22:19:58.674758 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:19:58.674730    4473 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5fb7086a-96b1-4b1c-a86d-8e38bca4cf80" containerName="main"
Oct 30 22:19:58.674758 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:19:58.674754    4473 cpu_manager.go:414] "containerMap: removing container" podUID="35fcc100-bc0e-4c7a-9e34-aa06b86f288a" containerName="payload-processing"
Oct 30 22:19:58.674758 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:19:58.674761    4473 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="35fcc100-bc0e-4c7a-9e34-aa06b86f288a" containerName="payload-processing"
Oct 30 22:19:58.674965 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:19:58.674768    4473 cpu_manager.go:414] "containerMap: removing container" podUID="706bd83a-953d-4b2e-8f1c-e51bf4343a19" containerName="payload-pre-processing"
Oct 30 22:19:58.674965 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:19:58.674774    4473 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="706bd83a-953d-4b2e-8f1c-e51bf4343a19" containerName="payload-pre-processing"
Oct 30 22:19:58.674965 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:19:58.674785    4473 cpu_manager.go:414] "containerMap: removing container" podUID="7665007f-2e07-47f7-ac42-82dfd53d85a1" containerName="payload-pre-processing"
Oct 30 22:19:58.674965 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:19:58.674792    4473 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7665007f-2e07-47f7-ac42-82dfd53d85a1" containerName="payload-pre-processing"
Oct 30 22:19:58.674965 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:19:58.674804    4473 cpu_manager.go:414] "containerMap: removing container" podUID="40fb21f2-c4df-4bfd-b6e9-4198045570be" containerName="payload-pre-processing"
Oct 30 22:19:58.674965 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:19:58.674808    4473 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="40fb21f2-c4df-4bfd-b6e9-4198045570be" containerName="payload-pre-processing"
Oct 30 22:19:58.674965 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:19:58.674829    4473 cpu_manager.go:414] "containerMap: removing container" podUID="0dcc6ba9-67e6-4ec3-9315-305edff7967e" containerName="maas-api"
Oct 30 22:19:58.674965 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:19:58.674834    4473 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0dcc6ba9-67e6-4ec3-9315-305edff7967e" containerName="maas-api"
Oct 30 22:19:58.674965 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:19:58.674843    4473 cpu_manager.go:414] "containerMap: removing container" podUID="0b139aed-161c-425c-a63c-59441bcdaa54" containerName="revoke-keys"
Oct 30 22:19:58.674965 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:19:58.674847    4473 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0b139aed-161c-425c-a63c-59441bcdaa54" containerName="revoke-keys"
Oct 30 22:19:58.674965 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:19:58.674855    4473 cpu_manager.go:414] "containerMap: removing container" podUID="f46a30dc-0abe-4906-8b58-c439a0bff711" containerName="istio-proxy"
Oct 30 22:19:58.674965 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:19:58.674859    4473 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f46a30dc-0abe-4906-8b58-c439a0bff711" containerName="istio-proxy"
Oct 30 22:19:58.674965 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:19:58.674871    4473 cpu_manager.go:414] "containerMap: removing container" podUID="b372cc09-4720-4855-9c3a-0e8e7b8beb70" containerName="maas-api"
Oct 30 22:19:58.674965 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:19:58.674875    4473 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b372cc09-4720-4855-9c3a-0e8e7b8beb70" containerName="maas-api"
Oct 30 22:19:58.674965 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:19:58.674883    4473 cpu_manager.go:414] "containerMap: removing container" podUID="cd728188-2246-4f1d-8f18-ec351a96b935" containerName="payload-processing"
Oct 30 22:19:58.674965 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:19:58.674888    4473 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cd728188-2246-4f1d-8f18-ec351a96b935" containerName="payload-processing"
Oct 30 22:19:58.675441 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:19:58.674996    4473 memory_manager.go:365] "RemoveStaleState removing state" podUID="5fb7086a-96b1-4b1c-a86d-8e38bca4cf80" containerName="main"
Oct 30 22:19:58.675441 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:19:58.675006    4473 memory_manager.go:365] "RemoveStaleState removing state" podUID="35fcc100-bc0e-4c7a-9e34-aa06b86f288a" containerName="payload-processing"
Oct 30 22:19:58.675441 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:19:58.675011    4473 memory_manager.go:365] "RemoveStaleState removing state" podUID="706bd83a-953d-4b2e-8f1c-e51bf4343a19" containerName="payload-pre-processing"
Oct 30 22:19:58.675441 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:19:58.675017    4473 memory_manager.go:365] "RemoveStaleState removing state" podUID="7665007f-2e07-47f7-ac42-82dfd53d85a1" containerName="payload-pre-processing"
Oct 30 22:19:58.675441 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:19:58.675023    4473 memory_manager.go:365] "RemoveStaleState removing state" podUID="40fb21f2-c4df-4bfd-b6e9-4198045570be" containerName="payload-pre-processing"
Oct 30 22:19:58.675441 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:19:58.675030    4473 memory_manager.go:365] "RemoveStaleState removing state" podUID="0dcc6ba9-67e6-4ec3-9315-305edff7967e" containerName="maas-api"
Oct 30 22:19:58.675441 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:19:58.675036    4473 memory_manager.go:365] "RemoveStaleState removing state" podUID="0b139aed-161c-425c-a63c-59441bcdaa54" containerName="revoke-keys"
Oct 30 22:19:58.675441 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:19:58.675040    4473 memory_manager.go:365] "RemoveStaleState removing state" podUID="f46a30dc-0abe-4906-8b58-c439a0bff711" containerName="istio-proxy"
Oct 30 22:19:58.675441 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:19:58.675047    4473 memory_manager.go:365] "RemoveStaleState removing state" podUID="b372cc09-4720-4855-9c3a-0e8e7b8beb70" containerName="maas-api"
Oct 30 22:19:58.675441 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:19:58.675051    4473 memory_manager.go:365] "RemoveStaleState removing state" podUID="cd728188-2246-4f1d-8f18-ec351a96b935" containerName="payload-processing"
Oct 30 22:19:58.677827 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:19:58.677813    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6826e712-c97kp"
Oct 30 22:19:58.717863 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:19:58.717828    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-scvs7\" (UniqueName: \"kubernetes.io/projected/0df24205-ae49-4abd-abf8-afa8955f7d61-kube-api-access-scvs7\") pod \"maas-api-revoke-sub-6826e712-c97kp\" (UID: \"0df24205-ae49-4abd-abf8-afa8955f7d61\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6826e712-c97kp"
Oct 30 22:19:58.717863 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:19:58.717867    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/0df24205-ae49-4abd-abf8-afa8955f7d61-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-6826e712-c97kp\" (UID: \"0df24205-ae49-4abd-abf8-afa8955f7d61\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6826e712-c97kp"
Oct 30 22:19:58.718096 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:19:58.717898    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/0df24205-ae49-4abd-abf8-afa8955f7d61-maas-api-xks-ca\") pod \"maas-api-revoke-sub-6826e712-c97kp\" (UID: \"0df24205-ae49-4abd-abf8-afa8955f7d61\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6826e712-c97kp"
Oct 30 22:19:58.734022 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:19:58.733972    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-6826e712-c97kp"]
Oct 30 22:19:58.781851 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:19:58.781810    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 22:19:58.782047 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:19:58.781854    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-d786g\""
Oct 30 22:19:58.819137 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:19:58.819097    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-scvs7\" (UniqueName: \"kubernetes.io/projected/0df24205-ae49-4abd-abf8-afa8955f7d61-kube-api-access-scvs7\") pod \"maas-api-revoke-sub-6826e712-c97kp\" (UID: \"0df24205-ae49-4abd-abf8-afa8955f7d61\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6826e712-c97kp"
Oct 30 22:19:58.819137 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:19:58.819135    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/0df24205-ae49-4abd-abf8-afa8955f7d61-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-6826e712-c97kp\" (UID: \"0df24205-ae49-4abd-abf8-afa8955f7d61\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6826e712-c97kp"
Oct 30 22:19:58.819421 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:19:58.819180    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/0df24205-ae49-4abd-abf8-afa8955f7d61-maas-api-xks-ca\") pod \"maas-api-revoke-sub-6826e712-c97kp\" (UID: \"0df24205-ae49-4abd-abf8-afa8955f7d61\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6826e712-c97kp"
Oct 30 22:19:58.819807 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:19:58.819778    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/0df24205-ae49-4abd-abf8-afa8955f7d61-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-6826e712-c97kp\" (UID: \"0df24205-ae49-4abd-abf8-afa8955f7d61\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6826e712-c97kp"
Oct 30 22:19:58.821848 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:19:58.821829    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/0df24205-ae49-4abd-abf8-afa8955f7d61-maas-api-xks-ca\") pod \"maas-api-revoke-sub-6826e712-c97kp\" (UID: \"0df24205-ae49-4abd-abf8-afa8955f7d61\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6826e712-c97kp"
Oct 30 22:19:58.877198 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:19:58.877159    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-scvs7\" (UniqueName: \"kubernetes.io/projected/0df24205-ae49-4abd-abf8-afa8955f7d61-kube-api-access-scvs7\") pod \"maas-api-revoke-sub-6826e712-c97kp\" (UID: \"0df24205-ae49-4abd-abf8-afa8955f7d61\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6826e712-c97kp"
Oct 30 22:19:58.992386 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:19:58.992307    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6826e712-c97kp"
Oct 30 22:19:59.366027 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:19:59.366003    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-6826e712-c97kp"]
Oct 30 22:19:59.367993 ip-10-0-140-18 kubenswrapper[4473]: W1030 22:19:59.367912    4473 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0df24205_ae49_4abd_abf8_afa8955f7d61.slice/crio-688ae5fff9ad2f9199beaf1e79bd0250c97228b9062dc40d736fea6f4390463c WatchSource:0}: Error finding container 688ae5fff9ad2f9199beaf1e79bd0250c97228b9062dc40d736fea6f4390463c: Status 404 returned error can't find the container with id 688ae5fff9ad2f9199beaf1e79bd0250c97228b9062dc40d736fea6f4390463c
Oct 30 22:19:59.545372 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:19:59.545331    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6826e712-c97kp" event={"ID":"0df24205-ae49-4abd-abf8-afa8955f7d61","Type":"ContainerStarted","Data":"688ae5fff9ad2f9199beaf1e79bd0250c97228b9062dc40d736fea6f4390463c"}
Oct 30 22:20:00.197237 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:20:00.197201    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-242ddaf0-openshift-default-79b7ddb6f7-vxx64"]
Oct 30 22:20:00.197659 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:20:00.197407    4473 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-242ddaf0-openshift-default-79b7ddb6f7-vxx64" podUID="926d7274-b7f3-4a98-ab8b-c8dbe1178f36" containerName="istio-proxy" containerID="cri-o://c49c685014bb88936bdd56ed7f19bdec39e10af6dd170e2fcd7ad86157cf58ae" gracePeriod=30
Oct 30 22:20:00.551993 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:20:00.551896    4473 generic.go:356] "Generic (PLEG): container finished" podID="0df24205-ae49-4abd-abf8-afa8955f7d61" containerID="17a968ba4f8f54f0d8920c412953a83651300b237a11942abcb557c7b47e29dc" exitCode=0
Oct 30 22:20:00.552136 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:20:00.551995    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6826e712-c97kp" event={"ID":"0df24205-ae49-4abd-abf8-afa8955f7d61","Type":"ContainerDied","Data":"17a968ba4f8f54f0d8920c412953a83651300b237a11942abcb557c7b47e29dc"}
Oct 30 22:20:01.696455 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:20:01.696429    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6826e712-c97kp"
Oct 30 22:20:01.745433 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:20:01.745395    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0df24205-ae49-4abd-abf8-afa8955f7d61-kube-api-access-scvs7\" (UniqueName: \"kubernetes.io/projected/0df24205-ae49-4abd-abf8-afa8955f7d61-kube-api-access-scvs7\") pod \"0df24205-ae49-4abd-abf8-afa8955f7d61\" (UID: \"0df24205-ae49-4abd-abf8-afa8955f7d61\") "
Oct 30 22:20:01.745633 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:20:01.745443    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0df24205-ae49-4abd-abf8-afa8955f7d61-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/0df24205-ae49-4abd-abf8-afa8955f7d61-maas-api-xks-ca\") pod \"0df24205-ae49-4abd-abf8-afa8955f7d61\" (UID: \"0df24205-ae49-4abd-abf8-afa8955f7d61\") "
Oct 30 22:20:01.745633 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:20:01.745597    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0df24205-ae49-4abd-abf8-afa8955f7d61-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/0df24205-ae49-4abd-abf8-afa8955f7d61-maas-api-openshift-ca\") pod \"0df24205-ae49-4abd-abf8-afa8955f7d61\" (UID: \"0df24205-ae49-4abd-abf8-afa8955f7d61\") "
Oct 30 22:20:01.745991 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:20:01.745970    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0df24205-ae49-4abd-abf8-afa8955f7d61-maas-api-openshift-ca" pod "0df24205-ae49-4abd-abf8-afa8955f7d61" (UID: "0df24205-ae49-4abd-abf8-afa8955f7d61"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:20:01.747792 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:20:01.747773    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0df24205-ae49-4abd-abf8-afa8955f7d61-maas-api-xks-ca" pod "0df24205-ae49-4abd-abf8-afa8955f7d61" (UID: "0df24205-ae49-4abd-abf8-afa8955f7d61"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:20:01.747894 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:20:01.747873    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0df24205-ae49-4abd-abf8-afa8955f7d61-kube-api-access-scvs7" pod "0df24205-ae49-4abd-abf8-afa8955f7d61" (UID: "0df24205-ae49-4abd-abf8-afa8955f7d61"). InnerVolumeSpecName "kube-api-access-scvs7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:20:01.846489 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:20:01.846396    4473 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-scvs7\" (UniqueName: \"kubernetes.io/projected/0df24205-ae49-4abd-abf8-afa8955f7d61-kube-api-access-scvs7\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:20:01.846489 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:20:01.846428    4473 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/0df24205-ae49-4abd-abf8-afa8955f7d61-maas-api-xks-ca\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:20:01.846489 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:20:01.846441    4473 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/0df24205-ae49-4abd-abf8-afa8955f7d61-maas-api-openshift-ca\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:20:02.562708 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:20:02.562670    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6826e712-c97kp"
Oct 30 22:20:02.562877 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:20:02.562701    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6826e712-c97kp" event={"ID":"0df24205-ae49-4abd-abf8-afa8955f7d61","Type":"ContainerDied","Data":"688ae5fff9ad2f9199beaf1e79bd0250c97228b9062dc40d736fea6f4390463c"}
Oct 30 22:20:02.562877 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:20:02.562756    4473 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="688ae5fff9ad2f9199beaf1e79bd0250c97228b9062dc40d736fea6f4390463c"
Oct 30 22:20:05.446858 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:20:05.446833    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-242ddaf0-openshift-default-79b7ddb6f7-vxx64"
Oct 30 22:20:05.476107 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:20:05.476024    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/926d7274-b7f3-4a98-ab8b-c8dbe1178f36-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/926d7274-b7f3-4a98-ab8b-c8dbe1178f36-workload-socket\") pod \"926d7274-b7f3-4a98-ab8b-c8dbe1178f36\" (UID: \"926d7274-b7f3-4a98-ab8b-c8dbe1178f36\") "
Oct 30 22:20:05.476107 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:20:05.476104    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/926d7274-b7f3-4a98-ab8b-c8dbe1178f36-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/926d7274-b7f3-4a98-ab8b-c8dbe1178f36-istiod-ca-cert\") pod \"926d7274-b7f3-4a98-ab8b-c8dbe1178f36\" (UID: \"926d7274-b7f3-4a98-ab8b-c8dbe1178f36\") "
Oct 30 22:20:05.476306 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:20:05.476124    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/926d7274-b7f3-4a98-ab8b-c8dbe1178f36-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/926d7274-b7f3-4a98-ab8b-c8dbe1178f36-istio-podinfo\") pod \"926d7274-b7f3-4a98-ab8b-c8dbe1178f36\" (UID: \"926d7274-b7f3-4a98-ab8b-c8dbe1178f36\") "
Oct 30 22:20:05.476306 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:20:05.476143    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/926d7274-b7f3-4a98-ab8b-c8dbe1178f36-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/926d7274-b7f3-4a98-ab8b-c8dbe1178f36-credential-socket\") pod \"926d7274-b7f3-4a98-ab8b-c8dbe1178f36\" (UID: \"926d7274-b7f3-4a98-ab8b-c8dbe1178f36\") "
Oct 30 22:20:05.476306 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:20:05.476174    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/926d7274-b7f3-4a98-ab8b-c8dbe1178f36-istio-token\" (UniqueName: \"kubernetes.io/projected/926d7274-b7f3-4a98-ab8b-c8dbe1178f36-istio-token\") pod \"926d7274-b7f3-4a98-ab8b-c8dbe1178f36\" (UID: \"926d7274-b7f3-4a98-ab8b-c8dbe1178f36\") "
Oct 30 22:20:05.476306 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:20:05.476212    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/926d7274-b7f3-4a98-ab8b-c8dbe1178f36-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/926d7274-b7f3-4a98-ab8b-c8dbe1178f36-workload-certs\") pod \"926d7274-b7f3-4a98-ab8b-c8dbe1178f36\" (UID: \"926d7274-b7f3-4a98-ab8b-c8dbe1178f36\") "
Oct 30 22:20:05.476306 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:20:05.476274    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/926d7274-b7f3-4a98-ab8b-c8dbe1178f36-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/926d7274-b7f3-4a98-ab8b-c8dbe1178f36-istio-envoy\") pod \"926d7274-b7f3-4a98-ab8b-c8dbe1178f36\" (UID: \"926d7274-b7f3-4a98-ab8b-c8dbe1178f36\") "
Oct 30 22:20:05.476566 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:20:05.476342    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/926d7274-b7f3-4a98-ab8b-c8dbe1178f36-workload-socket" pod "926d7274-b7f3-4a98-ab8b-c8dbe1178f36" (UID: "926d7274-b7f3-4a98-ab8b-c8dbe1178f36"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:20:05.476566 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:20:05.476477    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/926d7274-b7f3-4a98-ab8b-c8dbe1178f36-credential-socket" pod "926d7274-b7f3-4a98-ab8b-c8dbe1178f36" (UID: "926d7274-b7f3-4a98-ab8b-c8dbe1178f36"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:20:05.476663 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:20:05.476601    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/926d7274-b7f3-4a98-ab8b-c8dbe1178f36-workload-certs" pod "926d7274-b7f3-4a98-ab8b-c8dbe1178f36" (UID: "926d7274-b7f3-4a98-ab8b-c8dbe1178f36"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:20:05.476663 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:20:05.476617    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/926d7274-b7f3-4a98-ab8b-c8dbe1178f36-istiod-ca-cert" pod "926d7274-b7f3-4a98-ab8b-c8dbe1178f36" (UID: "926d7274-b7f3-4a98-ab8b-c8dbe1178f36"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:20:05.476741 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:20:05.476722    4473 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/926d7274-b7f3-4a98-ab8b-c8dbe1178f36-istiod-ca-cert\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:20:05.476741 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:20:05.476736    4473 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/926d7274-b7f3-4a98-ab8b-c8dbe1178f36-credential-socket\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:20:05.476814 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:20:05.476745    4473 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/926d7274-b7f3-4a98-ab8b-c8dbe1178f36-workload-certs\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:20:05.476814 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:20:05.476754    4473 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/926d7274-b7f3-4a98-ab8b-c8dbe1178f36-workload-socket\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:20:05.478550 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:20:05.478531    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/926d7274-b7f3-4a98-ab8b-c8dbe1178f36-istio-envoy" pod "926d7274-b7f3-4a98-ab8b-c8dbe1178f36" (UID: "926d7274-b7f3-4a98-ab8b-c8dbe1178f36"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:20:05.478989 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:20:05.478972    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/926d7274-b7f3-4a98-ab8b-c8dbe1178f36-istio-token" pod "926d7274-b7f3-4a98-ab8b-c8dbe1178f36" (UID: "926d7274-b7f3-4a98-ab8b-c8dbe1178f36"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:20:05.478989 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:20:05.478978    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/926d7274-b7f3-4a98-ab8b-c8dbe1178f36-istio-podinfo" pod "926d7274-b7f3-4a98-ab8b-c8dbe1178f36" (UID: "926d7274-b7f3-4a98-ab8b-c8dbe1178f36"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:20:05.577355 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:20:05.577318    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/926d7274-b7f3-4a98-ab8b-c8dbe1178f36-kube-api-access-rrknj\" (UniqueName: \"kubernetes.io/projected/926d7274-b7f3-4a98-ab8b-c8dbe1178f36-kube-api-access-rrknj\") pod \"926d7274-b7f3-4a98-ab8b-c8dbe1178f36\" (UID: \"926d7274-b7f3-4a98-ab8b-c8dbe1178f36\") "
Oct 30 22:20:05.577529 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:20:05.577367    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/926d7274-b7f3-4a98-ab8b-c8dbe1178f36-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/926d7274-b7f3-4a98-ab8b-c8dbe1178f36-istio-data\") pod \"926d7274-b7f3-4a98-ab8b-c8dbe1178f36\" (UID: \"926d7274-b7f3-4a98-ab8b-c8dbe1178f36\") "
Oct 30 22:20:05.577710 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:20:05.577689    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/926d7274-b7f3-4a98-ab8b-c8dbe1178f36-istio-data" pod "926d7274-b7f3-4a98-ab8b-c8dbe1178f36" (UID: "926d7274-b7f3-4a98-ab8b-c8dbe1178f36"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:20:05.577789 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:20:05.577754    4473 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/926d7274-b7f3-4a98-ab8b-c8dbe1178f36-istio-token\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:20:05.577789 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:20:05.577768    4473 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/926d7274-b7f3-4a98-ab8b-c8dbe1178f36-istio-envoy\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:20:05.577789 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:20:05.577778    4473 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/926d7274-b7f3-4a98-ab8b-c8dbe1178f36-istio-podinfo\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:20:05.578837 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:20:05.578812    4473 generic.go:356] "Generic (PLEG): container finished" podID="926d7274-b7f3-4a98-ab8b-c8dbe1178f36" containerID="c49c685014bb88936bdd56ed7f19bdec39e10af6dd170e2fcd7ad86157cf58ae" exitCode=0
Oct 30 22:20:05.578908 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:20:05.578896    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-242ddaf0-openshift-default-79b7ddb6f7-vxx64" event={"ID":"926d7274-b7f3-4a98-ab8b-c8dbe1178f36","Type":"ContainerDied","Data":"c49c685014bb88936bdd56ed7f19bdec39e10af6dd170e2fcd7ad86157cf58ae"}
Oct 30 22:20:05.578983 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:20:05.578912    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-242ddaf0-openshift-default-79b7ddb6f7-vxx64"
Oct 30 22:20:05.578983 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:20:05.578972    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-242ddaf0-openshift-default-79b7ddb6f7-vxx64" event={"ID":"926d7274-b7f3-4a98-ab8b-c8dbe1178f36","Type":"ContainerDied","Data":"3ceff137d44b351e22010d09b05192ce61f495140a48e618e3410ee464b8c429"}
Oct 30 22:20:05.579057 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:20:05.579005    4473 scope.go:122] "RemoveContainer" containerID="c49c685014bb88936bdd56ed7f19bdec39e10af6dd170e2fcd7ad86157cf58ae"
Oct 30 22:20:05.580047 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:20:05.579960    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/926d7274-b7f3-4a98-ab8b-c8dbe1178f36-kube-api-access-rrknj" pod "926d7274-b7f3-4a98-ab8b-c8dbe1178f36" (UID: "926d7274-b7f3-4a98-ab8b-c8dbe1178f36"). InnerVolumeSpecName "kube-api-access-rrknj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:20:05.588374 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:20:05.588355    4473 scope.go:122] "RemoveContainer" containerID="c49c685014bb88936bdd56ed7f19bdec39e10af6dd170e2fcd7ad86157cf58ae"
Oct 30 22:20:05.588667 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:20:05.588645    4473 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c49c685014bb88936bdd56ed7f19bdec39e10af6dd170e2fcd7ad86157cf58ae\": container with ID starting with c49c685014bb88936bdd56ed7f19bdec39e10af6dd170e2fcd7ad86157cf58ae not found: ID does not exist" containerID="c49c685014bb88936bdd56ed7f19bdec39e10af6dd170e2fcd7ad86157cf58ae"
Oct 30 22:20:05.588727 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:20:05.588680    4473 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c49c685014bb88936bdd56ed7f19bdec39e10af6dd170e2fcd7ad86157cf58ae"} err="failed to get container status \"c49c685014bb88936bdd56ed7f19bdec39e10af6dd170e2fcd7ad86157cf58ae\": rpc error: code = NotFound desc = could not find container \"c49c685014bb88936bdd56ed7f19bdec39e10af6dd170e2fcd7ad86157cf58ae\": container with ID starting with c49c685014bb88936bdd56ed7f19bdec39e10af6dd170e2fcd7ad86157cf58ae not found: ID does not exist"
Oct 30 22:20:05.678571 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:20:05.678529    4473 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rrknj\" (UniqueName: \"kubernetes.io/projected/926d7274-b7f3-4a98-ab8b-c8dbe1178f36-kube-api-access-rrknj\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:20:05.678571 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:20:05.678571    4473 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/926d7274-b7f3-4a98-ab8b-c8dbe1178f36-istio-data\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:20:06.001635 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:20:06.001595    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-242ddaf0-openshift-default-79b7ddb6f7-vxx64"]
Oct 30 22:20:06.005765 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:20:06.005737    4473 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-242ddaf0-openshift-default-79b7ddb6f7-vxx64"]
Oct 30 22:20:06.042631 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:20:06.042592    4473 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w6-29cbad/premium-w6-29cbad-kserve-6c9f8d6899-qtv6c" podUID="381b11f6-3fde-4de7-9ad3-7c0891e76ba1" containerName="main" containerID="cri-o://4bfc1fcafc15d27d5fc2e5618f2fc39247300c9d78255d4ee6935452201c747f" gracePeriod=45
Oct 30 22:20:06.296664 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:20:06.296638    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-29cbad/premium-w6-29cbad-kserve-6c9f8d6899-qtv6c"
Oct 30 22:20:06.355888 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:20:06.355858    4473 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="926d7274-b7f3-4a98-ab8b-c8dbe1178f36" path="/var/lib/kubelet/pods/926d7274-b7f3-4a98-ab8b-c8dbe1178f36/volumes"
Oct 30 22:20:06.385143 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:20:06.385109    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/381b11f6-3fde-4de7-9ad3-7c0891e76ba1-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/381b11f6-3fde-4de7-9ad3-7c0891e76ba1-tmp-dir\") pod \"381b11f6-3fde-4de7-9ad3-7c0891e76ba1\" (UID: \"381b11f6-3fde-4de7-9ad3-7c0891e76ba1\") "
Oct 30 22:20:06.385324 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:20:06.385189    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/381b11f6-3fde-4de7-9ad3-7c0891e76ba1-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/381b11f6-3fde-4de7-9ad3-7c0891e76ba1-model-cache\") pod \"381b11f6-3fde-4de7-9ad3-7c0891e76ba1\" (UID: \"381b11f6-3fde-4de7-9ad3-7c0891e76ba1\") "
Oct 30 22:20:06.385324 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:20:06.385241    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/381b11f6-3fde-4de7-9ad3-7c0891e76ba1-home\" (UniqueName: \"kubernetes.io/empty-dir/381b11f6-3fde-4de7-9ad3-7c0891e76ba1-home\") pod \"381b11f6-3fde-4de7-9ad3-7c0891e76ba1\" (UID: \"381b11f6-3fde-4de7-9ad3-7c0891e76ba1\") "
Oct 30 22:20:06.385324 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:20:06.385275    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/381b11f6-3fde-4de7-9ad3-7c0891e76ba1-kube-api-access-d5hwj\" (UniqueName: \"kubernetes.io/projected/381b11f6-3fde-4de7-9ad3-7c0891e76ba1-kube-api-access-d5hwj\") pod \"381b11f6-3fde-4de7-9ad3-7c0891e76ba1\" (UID: \"381b11f6-3fde-4de7-9ad3-7c0891e76ba1\") "
Oct 30 22:20:06.385481 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:20:06.385337    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/381b11f6-3fde-4de7-9ad3-7c0891e76ba1-dshm\" (UniqueName: \"kubernetes.io/empty-dir/381b11f6-3fde-4de7-9ad3-7c0891e76ba1-dshm\") pod \"381b11f6-3fde-4de7-9ad3-7c0891e76ba1\" (UID: \"381b11f6-3fde-4de7-9ad3-7c0891e76ba1\") "
Oct 30 22:20:06.385481 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:20:06.385375    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/381b11f6-3fde-4de7-9ad3-7c0891e76ba1-tls-certs\" (UniqueName: \"kubernetes.io/secret/381b11f6-3fde-4de7-9ad3-7c0891e76ba1-tls-certs\") pod \"381b11f6-3fde-4de7-9ad3-7c0891e76ba1\" (UID: \"381b11f6-3fde-4de7-9ad3-7c0891e76ba1\") "
Oct 30 22:20:06.385481 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:20:06.385377    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/381b11f6-3fde-4de7-9ad3-7c0891e76ba1-tmp-dir" pod "381b11f6-3fde-4de7-9ad3-7c0891e76ba1" (UID: "381b11f6-3fde-4de7-9ad3-7c0891e76ba1"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:20:06.385659 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:20:06.385500    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/381b11f6-3fde-4de7-9ad3-7c0891e76ba1-model-cache" pod "381b11f6-3fde-4de7-9ad3-7c0891e76ba1" (UID: "381b11f6-3fde-4de7-9ad3-7c0891e76ba1"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:20:06.385659 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:20:06.385558    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/381b11f6-3fde-4de7-9ad3-7c0891e76ba1-home" pod "381b11f6-3fde-4de7-9ad3-7c0891e76ba1" (UID: "381b11f6-3fde-4de7-9ad3-7c0891e76ba1"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:20:06.385775 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:20:06.385759    4473 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/381b11f6-3fde-4de7-9ad3-7c0891e76ba1-tmp-dir\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:20:06.385821 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:20:06.385779    4473 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/381b11f6-3fde-4de7-9ad3-7c0891e76ba1-model-cache\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:20:06.385821 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:20:06.385787    4473 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/381b11f6-3fde-4de7-9ad3-7c0891e76ba1-home\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:20:06.387800 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:20:06.387770    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/381b11f6-3fde-4de7-9ad3-7c0891e76ba1-tls-certs" pod "381b11f6-3fde-4de7-9ad3-7c0891e76ba1" (UID: "381b11f6-3fde-4de7-9ad3-7c0891e76ba1"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:20:06.388108 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:20:06.388085    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/381b11f6-3fde-4de7-9ad3-7c0891e76ba1-dshm" pod "381b11f6-3fde-4de7-9ad3-7c0891e76ba1" (UID: "381b11f6-3fde-4de7-9ad3-7c0891e76ba1"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:20:06.388181 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:20:06.388105    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/381b11f6-3fde-4de7-9ad3-7c0891e76ba1-kube-api-access-d5hwj" pod "381b11f6-3fde-4de7-9ad3-7c0891e76ba1" (UID: "381b11f6-3fde-4de7-9ad3-7c0891e76ba1"). InnerVolumeSpecName "kube-api-access-d5hwj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:20:06.486888 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:20:06.486776    4473 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d5hwj\" (UniqueName: \"kubernetes.io/projected/381b11f6-3fde-4de7-9ad3-7c0891e76ba1-kube-api-access-d5hwj\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:20:06.486888 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:20:06.486816    4473 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/381b11f6-3fde-4de7-9ad3-7c0891e76ba1-dshm\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:20:06.486888 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:20:06.486831    4473 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/381b11f6-3fde-4de7-9ad3-7c0891e76ba1-tls-certs\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:20:06.586353 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:20:06.586315    4473 generic.go:356] "Generic (PLEG): container finished" podID="381b11f6-3fde-4de7-9ad3-7c0891e76ba1" containerID="4bfc1fcafc15d27d5fc2e5618f2fc39247300c9d78255d4ee6935452201c747f" exitCode=0
Oct 30 22:20:06.586353 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:20:06.586353    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-29cbad/premium-w6-29cbad-kserve-6c9f8d6899-qtv6c"
Oct 30 22:20:06.586581 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:20:06.586393    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-29cbad/premium-w6-29cbad-kserve-6c9f8d6899-qtv6c" event={"ID":"381b11f6-3fde-4de7-9ad3-7c0891e76ba1","Type":"ContainerDied","Data":"4bfc1fcafc15d27d5fc2e5618f2fc39247300c9d78255d4ee6935452201c747f"}
Oct 30 22:20:06.586581 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:20:06.586436    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-29cbad/premium-w6-29cbad-kserve-6c9f8d6899-qtv6c" event={"ID":"381b11f6-3fde-4de7-9ad3-7c0891e76ba1","Type":"ContainerDied","Data":"a079d20c320af5c38c2e26ab1736f16a9bc42b6793b728b7d62ff8f25e0104ed"}
Oct 30 22:20:06.586581 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:20:06.586458    4473 scope.go:122] "RemoveContainer" containerID="4bfc1fcafc15d27d5fc2e5618f2fc39247300c9d78255d4ee6935452201c747f"
Oct 30 22:20:06.598676 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:20:06.598653    4473 scope.go:122] "RemoveContainer" containerID="4bfc1fcafc15d27d5fc2e5618f2fc39247300c9d78255d4ee6935452201c747f"
Oct 30 22:20:06.599026 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:20:06.599008    4473 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4bfc1fcafc15d27d5fc2e5618f2fc39247300c9d78255d4ee6935452201c747f\": container with ID starting with 4bfc1fcafc15d27d5fc2e5618f2fc39247300c9d78255d4ee6935452201c747f not found: ID does not exist" containerID="4bfc1fcafc15d27d5fc2e5618f2fc39247300c9d78255d4ee6935452201c747f"
Oct 30 22:20:06.599107 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:20:06.599038    4473 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4bfc1fcafc15d27d5fc2e5618f2fc39247300c9d78255d4ee6935452201c747f"} err="failed to get container status \"4bfc1fcafc15d27d5fc2e5618f2fc39247300c9d78255d4ee6935452201c747f\": rpc error: code = NotFound desc = could not find container \"4bfc1fcafc15d27d5fc2e5618f2fc39247300c9d78255d4ee6935452201c747f\": container with ID starting with 4bfc1fcafc15d27d5fc2e5618f2fc39247300c9d78255d4ee6935452201c747f not found: ID does not exist"
Oct 30 22:20:06.758859 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:20:06.758773    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w6-29cbad/premium-w6-29cbad-kserve-6c9f8d6899-qtv6c"]
Oct 30 22:20:06.765327 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:20:06.765301    4473 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w6-29cbad/premium-w6-29cbad-kserve-6c9f8d6899-qtv6c"]
Oct 30 22:20:08.360625 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:20:08.360577    4473 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="381b11f6-3fde-4de7-9ad3-7c0891e76ba1" path="/var/lib/kubelet/pods/381b11f6-3fde-4de7-9ad3-7c0891e76ba1/volumes"
Oct 30 22:20:20.324289 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:20:20.324248    4473 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fncvg" containerName="extractor"
Oct 30 22:20:28.203465 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:20:28.203427    4473 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-5b9r4"]
Oct 30 22:20:28.203943 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:20:28.203907    4473 cpu_manager.go:414] "containerMap: removing container" podUID="926d7274-b7f3-4a98-ab8b-c8dbe1178f36" containerName="istio-proxy"
Oct 30 22:20:28.203943 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:20:28.203938    4473 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="926d7274-b7f3-4a98-ab8b-c8dbe1178f36" containerName="istio-proxy"
Oct 30 22:20:28.204110 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:20:28.203962    4473 cpu_manager.go:414] "containerMap: removing container" podUID="0df24205-ae49-4abd-abf8-afa8955f7d61" containerName="revoke-keys"
Oct 30 22:20:28.204110 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:20:28.203970    4473 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0df24205-ae49-4abd-abf8-afa8955f7d61" containerName="revoke-keys"
Oct 30 22:20:28.204110 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:20:28.203990    4473 cpu_manager.go:414] "containerMap: removing container" podUID="381b11f6-3fde-4de7-9ad3-7c0891e76ba1" containerName="main"
Oct 30 22:20:28.204110 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:20:28.203996    4473 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="381b11f6-3fde-4de7-9ad3-7c0891e76ba1" containerName="main"
Oct 30 22:20:28.204110 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:20:28.204076    4473 memory_manager.go:365] "RemoveStaleState removing state" podUID="381b11f6-3fde-4de7-9ad3-7c0891e76ba1" containerName="main"
Oct 30 22:20:28.204110 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:20:28.204088    4473 memory_manager.go:365] "RemoveStaleState removing state" podUID="926d7274-b7f3-4a98-ab8b-c8dbe1178f36" containerName="istio-proxy"
Oct 30 22:20:28.204110 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:20:28.204097    4473 memory_manager.go:365] "RemoveStaleState removing state" podUID="0df24205-ae49-4abd-abf8-afa8955f7d61" containerName="revoke-keys"
Oct 30 22:20:28.208705 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:20:28.208685    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-5b9r4"
Oct 30 22:20:28.303196 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:20:28.303155    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-5b9r4"]
Oct 30 22:20:28.331287 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:20:28.331251    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1557635a-ac42-4385-99f3-879641eb0ccc-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-5b9r4\" (UID: \"1557635a-ac42-4385-99f3-879641eb0ccc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-5b9r4"
Oct 30 22:20:28.331474 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:20:28.331292    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1557635a-ac42-4385-99f3-879641eb0ccc-istio-data\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-5b9r4\" (UID: \"1557635a-ac42-4385-99f3-879641eb0ccc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-5b9r4"
Oct 30 22:20:28.331474 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:20:28.331382    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1557635a-ac42-4385-99f3-879641eb0ccc-workload-certs\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-5b9r4\" (UID: \"1557635a-ac42-4385-99f3-879641eb0ccc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-5b9r4"
Oct 30 22:20:28.331474 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:20:28.331464    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1557635a-ac42-4385-99f3-879641eb0ccc-istio-token\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-5b9r4\" (UID: \"1557635a-ac42-4385-99f3-879641eb0ccc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-5b9r4"
Oct 30 22:20:28.331584 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:20:28.331498    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-klzgr\" (UniqueName: \"kubernetes.io/projected/1557635a-ac42-4385-99f3-879641eb0ccc-kube-api-access-klzgr\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-5b9r4\" (UID: \"1557635a-ac42-4385-99f3-879641eb0ccc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-5b9r4"
Oct 30 22:20:28.331584 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:20:28.331546    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1557635a-ac42-4385-99f3-879641eb0ccc-credential-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-5b9r4\" (UID: \"1557635a-ac42-4385-99f3-879641eb0ccc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-5b9r4"
Oct 30 22:20:28.331649 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:20:28.331618    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1557635a-ac42-4385-99f3-879641eb0ccc-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-5b9r4\" (UID: \"1557635a-ac42-4385-99f3-879641eb0ccc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-5b9r4"
Oct 30 22:20:28.331649 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:20:28.331639    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1557635a-ac42-4385-99f3-879641eb0ccc-workload-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-5b9r4\" (UID: \"1557635a-ac42-4385-99f3-879641eb0ccc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-5b9r4"
Oct 30 22:20:28.331722 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:20:28.331672    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1557635a-ac42-4385-99f3-879641eb0ccc-istio-envoy\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-5b9r4\" (UID: \"1557635a-ac42-4385-99f3-879641eb0ccc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-5b9r4"
Oct 30 22:20:28.433201 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:20:28.433162    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-klzgr\" (UniqueName: \"kubernetes.io/projected/1557635a-ac42-4385-99f3-879641eb0ccc-kube-api-access-klzgr\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-5b9r4\" (UID: \"1557635a-ac42-4385-99f3-879641eb0ccc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-5b9r4"
Oct 30 22:20:28.433399 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:20:28.433210    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1557635a-ac42-4385-99f3-879641eb0ccc-credential-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-5b9r4\" (UID: \"1557635a-ac42-4385-99f3-879641eb0ccc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-5b9r4"
Oct 30 22:20:28.433399 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:20:28.433253    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1557635a-ac42-4385-99f3-879641eb0ccc-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-5b9r4\" (UID: \"1557635a-ac42-4385-99f3-879641eb0ccc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-5b9r4"
Oct 30 22:20:28.433399 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:20:28.433276    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1557635a-ac42-4385-99f3-879641eb0ccc-workload-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-5b9r4\" (UID: \"1557635a-ac42-4385-99f3-879641eb0ccc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-5b9r4"
Oct 30 22:20:28.433399 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:20:28.433304    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1557635a-ac42-4385-99f3-879641eb0ccc-istio-envoy\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-5b9r4\" (UID: \"1557635a-ac42-4385-99f3-879641eb0ccc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-5b9r4"
Oct 30 22:20:28.433399 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:20:28.433343    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1557635a-ac42-4385-99f3-879641eb0ccc-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-5b9r4\" (UID: \"1557635a-ac42-4385-99f3-879641eb0ccc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-5b9r4"
Oct 30 22:20:28.433399 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:20:28.433374    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1557635a-ac42-4385-99f3-879641eb0ccc-istio-data\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-5b9r4\" (UID: \"1557635a-ac42-4385-99f3-879641eb0ccc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-5b9r4"
Oct 30 22:20:28.433730 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:20:28.433430    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1557635a-ac42-4385-99f3-879641eb0ccc-workload-certs\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-5b9r4\" (UID: \"1557635a-ac42-4385-99f3-879641eb0ccc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-5b9r4"
Oct 30 22:20:28.433730 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:20:28.433483    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1557635a-ac42-4385-99f3-879641eb0ccc-istio-token\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-5b9r4\" (UID: \"1557635a-ac42-4385-99f3-879641eb0ccc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-5b9r4"
Oct 30 22:20:28.433850 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:20:28.433723    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1557635a-ac42-4385-99f3-879641eb0ccc-credential-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-5b9r4\" (UID: \"1557635a-ac42-4385-99f3-879641eb0ccc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-5b9r4"
Oct 30 22:20:28.433850 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:20:28.433804    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1557635a-ac42-4385-99f3-879641eb0ccc-istio-data\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-5b9r4\" (UID: \"1557635a-ac42-4385-99f3-879641eb0ccc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-5b9r4"
Oct 30 22:20:28.434023 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:20:28.433898    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1557635a-ac42-4385-99f3-879641eb0ccc-workload-certs\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-5b9r4\" (UID: \"1557635a-ac42-4385-99f3-879641eb0ccc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-5b9r4"
Oct 30 22:20:28.434130 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:20:28.434104    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1557635a-ac42-4385-99f3-879641eb0ccc-workload-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-5b9r4\" (UID: \"1557635a-ac42-4385-99f3-879641eb0ccc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-5b9r4"
Oct 30 22:20:28.434443 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:20:28.434207    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1557635a-ac42-4385-99f3-879641eb0ccc-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-5b9r4\" (UID: \"1557635a-ac42-4385-99f3-879641eb0ccc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-5b9r4"
Oct 30 22:20:28.435888 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:20:28.435866    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1557635a-ac42-4385-99f3-879641eb0ccc-istio-envoy\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-5b9r4\" (UID: \"1557635a-ac42-4385-99f3-879641eb0ccc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-5b9r4"
Oct 30 22:20:28.436055 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:20:28.436038    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1557635a-ac42-4385-99f3-879641eb0ccc-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-5b9r4\" (UID: \"1557635a-ac42-4385-99f3-879641eb0ccc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-5b9r4"
Oct 30 22:20:28.490791 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:20:28.490714    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1557635a-ac42-4385-99f3-879641eb0ccc-istio-token\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-5b9r4\" (UID: \"1557635a-ac42-4385-99f3-879641eb0ccc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-5b9r4"
Oct 30 22:20:28.490981 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:20:28.490849    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-klzgr\" (UniqueName: \"kubernetes.io/projected/1557635a-ac42-4385-99f3-879641eb0ccc-kube-api-access-klzgr\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-5b9r4\" (UID: \"1557635a-ac42-4385-99f3-879641eb0ccc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-5b9r4"
Oct 30 22:20:28.522426 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:20:28.522384    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-5b9r4"
Oct 30 22:20:28.960186 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:20:28.960153    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-5b9r4"]
Oct 30 22:20:28.961051 ip-10-0-140-18 kubenswrapper[4473]: W1030 22:20:28.961024    4473 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1557635a_ac42_4385_99f3_879641eb0ccc.slice/crio-9c5b2f72184e27ffd5522178b8e3add38b8c1333805d41224ecb10d14b099ea4 WatchSource:0}: Error finding container 9c5b2f72184e27ffd5522178b8e3add38b8c1333805d41224ecb10d14b099ea4: Status 404 returned error can't find the container with id 9c5b2f72184e27ffd5522178b8e3add38b8c1333805d41224ecb10d14b099ea4
Oct 30 22:20:28.963289 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:20:28.963255    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:20:28.963355 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:20:28.963325    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:20:28.963400 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:20:28.963354    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:20:29.705681 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:20:29.705640    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-5b9r4" event={"ID":"1557635a-ac42-4385-99f3-879641eb0ccc","Type":"ContainerStarted","Data":"99b88f0b4ecca406be62a59fb6b876fc97dd4c3e37c7c46d1903453ac6c4d777"}
Oct 30 22:20:29.706134 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:20:29.705704    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-5b9r4" event={"ID":"1557635a-ac42-4385-99f3-879641eb0ccc","Type":"ContainerStarted","Data":"9c5b2f72184e27ffd5522178b8e3add38b8c1333805d41224ecb10d14b099ea4"}
Oct 30 22:20:29.878050 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:20:29.877995    4473 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-5b9r4" podStartSLOduration=1.8779751999999998 podStartE2EDuration="1.8779752s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:20:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:20:29.877185548 +0000 UTC m=+2275.650554036" watchObservedRunningTime="2026-10-30 22:20:29.8779752 +0000 UTC m=+2275.651343686"
Oct 30 22:20:30.522628 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:20:30.522593    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-5b9r4"
Oct 30 22:20:30.524172 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:20:30.524144    4473 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-5b9r4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.146:15021/healthz/ready\": dial tcp 10.133.0.146:15021: connect: connection refused" start-of-body=
Oct 30 22:20:30.524294 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:20:30.524201    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-5b9r4" podUID="1557635a-ac42-4385-99f3-879641eb0ccc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.146:15021/healthz/ready\": dial tcp 10.133.0.146:15021: connect: connection refused"
Oct 30 22:20:31.523526 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:20:31.523486    4473 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-5b9r4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.146:15021/healthz/ready\": dial tcp 10.133.0.146:15021: connect: connection refused" start-of-body=
Oct 30 22:20:31.523971 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:20:31.523543    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-5b9r4" podUID="1557635a-ac42-4385-99f3-879641eb0ccc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.146:15021/healthz/ready\": dial tcp 10.133.0.146:15021: connect: connection refused"
Oct 30 22:20:32.523404 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:20:32.523364    4473 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-5b9r4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.146:15021/healthz/ready\": dial tcp 10.133.0.146:15021: connect: connection refused" start-of-body=
Oct 30 22:20:32.523604 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:20:32.523436    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-5b9r4" podUID="1557635a-ac42-4385-99f3-879641eb0ccc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.146:15021/healthz/ready\": dial tcp 10.133.0.146:15021: connect: connection refused"
Oct 30 22:20:33.522981 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:20:33.522936    4473 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-5b9r4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.146:15021/healthz/ready\": dial tcp 10.133.0.146:15021: connect: connection refused" start-of-body=
Oct 30 22:20:33.523176 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:20:33.523014    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-5b9r4" podUID="1557635a-ac42-4385-99f3-879641eb0ccc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.146:15021/healthz/ready\": dial tcp 10.133.0.146:15021: connect: connection refused"
Oct 30 22:20:34.523748 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:20:34.523706    4473 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-5b9r4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.146:15021/healthz/ready\": dial tcp 10.133.0.146:15021: connect: connection refused" start-of-body=
Oct 30 22:20:34.524218 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:20:34.523767    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-5b9r4" podUID="1557635a-ac42-4385-99f3-879641eb0ccc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.146:15021/healthz/ready\": dial tcp 10.133.0.146:15021: connect: connection refused"
Oct 30 22:20:35.523784 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:20:35.523742    4473 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-5b9r4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.146:15021/healthz/ready\": dial tcp 10.133.0.146:15021: connect: connection refused" start-of-body=
Oct 30 22:20:35.524212 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:20:35.523797    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-5b9r4" podUID="1557635a-ac42-4385-99f3-879641eb0ccc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.146:15021/healthz/ready\": dial tcp 10.133.0.146:15021: connect: connection refused"
Oct 30 22:20:36.523194 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:20:36.523157    4473 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-5b9r4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.146:15021/healthz/ready\": dial tcp 10.133.0.146:15021: connect: connection refused" start-of-body=
Oct 30 22:20:36.523371 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:20:36.523224    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-5b9r4" podUID="1557635a-ac42-4385-99f3-879641eb0ccc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.146:15021/healthz/ready\": dial tcp 10.133.0.146:15021: connect: connection refused"
Oct 30 22:20:37.418820 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:20:37.418784    4473 scope.go:122] "RemoveContainer" containerID="f3f9a29e2d0de8c767849797a860f4090da88b8631a0fdf0348916a47410da24"
Oct 30 22:20:37.527423 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:20:37.527389    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-5b9r4"
Oct 30 22:20:37.527701 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:20:37.527674    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-5b9r4"
Oct 30 22:20:37.528571 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:20:37.528550    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-5b9r4"
Oct 30 22:20:50.323956 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:20:50.323852    4473 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fncvg" containerName="extractor"
Oct 30 22:21:04.093851 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:04.092981    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-0f42cb/distinct-2-w3-0f42cb-kserve-fc477897b-jk8qc"]
Oct 30 22:21:04.119267 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:04.119200    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-0f42cb/distinct-w3-0f42cb-kserve-fb54fdd59-n6w4l"]
Oct 30 22:21:11.082593 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:11.082557    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-0f42cb-566b854747-xhdwq"]
Oct 30 22:21:14.986070 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:14.986035    4473 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-459d3dc2-r7c8g"]
Oct 30 22:21:14.992087 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:14.990231    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-459d3dc2-r7c8g"
Oct 30 22:21:15.048062 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:15.048027    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-459d3dc2-r7c8g"]
Oct 30 22:21:15.084388 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:15.084342    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2cv4z\" (UniqueName: \"kubernetes.io/projected/3e9f27bc-2bf5-4b5d-b6b1-d63c5271624e-kube-api-access-2cv4z\") pod \"maas-api-revoke-sub-459d3dc2-r7c8g\" (UID: \"3e9f27bc-2bf5-4b5d-b6b1-d63c5271624e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-459d3dc2-r7c8g"
Oct 30 22:21:15.084591 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:15.084505    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/3e9f27bc-2bf5-4b5d-b6b1-d63c5271624e-maas-api-xks-ca\") pod \"maas-api-revoke-sub-459d3dc2-r7c8g\" (UID: \"3e9f27bc-2bf5-4b5d-b6b1-d63c5271624e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-459d3dc2-r7c8g"
Oct 30 22:21:15.084591 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:15.084551    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/3e9f27bc-2bf5-4b5d-b6b1-d63c5271624e-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-459d3dc2-r7c8g\" (UID: \"3e9f27bc-2bf5-4b5d-b6b1-d63c5271624e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-459d3dc2-r7c8g"
Oct 30 22:21:15.095831 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:15.095798    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 22:21:15.096068 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:15.095840    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-d786g\""
Oct 30 22:21:15.185246 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:15.185211    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/3e9f27bc-2bf5-4b5d-b6b1-d63c5271624e-maas-api-xks-ca\") pod \"maas-api-revoke-sub-459d3dc2-r7c8g\" (UID: \"3e9f27bc-2bf5-4b5d-b6b1-d63c5271624e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-459d3dc2-r7c8g"
Oct 30 22:21:15.185413 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:15.185261    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/3e9f27bc-2bf5-4b5d-b6b1-d63c5271624e-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-459d3dc2-r7c8g\" (UID: \"3e9f27bc-2bf5-4b5d-b6b1-d63c5271624e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-459d3dc2-r7c8g"
Oct 30 22:21:15.185413 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:15.185325    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2cv4z\" (UniqueName: \"kubernetes.io/projected/3e9f27bc-2bf5-4b5d-b6b1-d63c5271624e-kube-api-access-2cv4z\") pod \"maas-api-revoke-sub-459d3dc2-r7c8g\" (UID: \"3e9f27bc-2bf5-4b5d-b6b1-d63c5271624e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-459d3dc2-r7c8g"
Oct 30 22:21:15.185913 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:15.185874    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/3e9f27bc-2bf5-4b5d-b6b1-d63c5271624e-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-459d3dc2-r7c8g\" (UID: \"3e9f27bc-2bf5-4b5d-b6b1-d63c5271624e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-459d3dc2-r7c8g"
Oct 30 22:21:15.187812 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:15.187789    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/3e9f27bc-2bf5-4b5d-b6b1-d63c5271624e-maas-api-xks-ca\") pod \"maas-api-revoke-sub-459d3dc2-r7c8g\" (UID: \"3e9f27bc-2bf5-4b5d-b6b1-d63c5271624e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-459d3dc2-r7c8g"
Oct 30 22:21:15.243407 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:15.243322    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2cv4z\" (UniqueName: \"kubernetes.io/projected/3e9f27bc-2bf5-4b5d-b6b1-d63c5271624e-kube-api-access-2cv4z\") pod \"maas-api-revoke-sub-459d3dc2-r7c8g\" (UID: \"3e9f27bc-2bf5-4b5d-b6b1-d63c5271624e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-459d3dc2-r7c8g"
Oct 30 22:21:15.307738 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:15.307702    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-459d3dc2-r7c8g"
Oct 30 22:21:15.682213 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:15.682188    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-459d3dc2-r7c8g"]
Oct 30 22:21:15.684352 ip-10-0-140-18 kubenswrapper[4473]: W1030 22:21:15.684324    4473 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e9f27bc_2bf5_4b5d_b6b1_d63c5271624e.slice/crio-027c3290ad1622fd2cab9a015eeebe0f29138030c742efe75fc95830a34e76dd WatchSource:0}: Error finding container 027c3290ad1622fd2cab9a015eeebe0f29138030c742efe75fc95830a34e76dd: Status 404 returned error can't find the container with id 027c3290ad1622fd2cab9a015eeebe0f29138030c742efe75fc95830a34e76dd
Oct 30 22:21:15.980339 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:15.980303    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-459d3dc2-r7c8g" event={"ID":"3e9f27bc-2bf5-4b5d-b6b1-d63c5271624e","Type":"ContainerStarted","Data":"2ca89a3ebd72788149df78b1b7805151a280ac71c2f4451d81336ff7354ba9a1"}
Oct 30 22:21:15.980523 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:15.980363    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-459d3dc2-r7c8g" event={"ID":"3e9f27bc-2bf5-4b5d-b6b1-d63c5271624e","Type":"ContainerStarted","Data":"027c3290ad1622fd2cab9a015eeebe0f29138030c742efe75fc95830a34e76dd"}
Oct 30 22:21:16.083895 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:16.083856    4473 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w3-0f42cb-566b854747-xhdwq" podUID="17ff1c66-c309-45da-9843-b37262d288d6" containerName="payload-processing" containerID="cri-o://c47c4349be6968d4cac79d5fd1241de583baae04f584f112f57c91281fca9054" gracePeriod=25
Oct 30 22:21:16.097990 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:16.097881    4473 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-459d3dc2-r7c8g" podStartSLOduration=2.097863259 podStartE2EDuration="2.097863259s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:21:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:21:16.096238405 +0000 UTC m=+2321.869606904" watchObservedRunningTime="2026-10-30 22:21:16.097863259 +0000 UTC m=+2321.871231746"
Oct 30 22:21:16.986395 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:16.986363    4473 generic.go:356] "Generic (PLEG): container finished" podID="3e9f27bc-2bf5-4b5d-b6b1-d63c5271624e" containerID="2ca89a3ebd72788149df78b1b7805151a280ac71c2f4451d81336ff7354ba9a1" exitCode=0
Oct 30 22:21:16.986583 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:16.986403    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-459d3dc2-r7c8g" event={"ID":"3e9f27bc-2bf5-4b5d-b6b1-d63c5271624e","Type":"ContainerDied","Data":"2ca89a3ebd72788149df78b1b7805151a280ac71c2f4451d81336ff7354ba9a1"}
Oct 30 22:21:18.131829 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:18.131804    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-459d3dc2-r7c8g"
Oct 30 22:21:18.219827 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:18.219796    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3e9f27bc-2bf5-4b5d-b6b1-d63c5271624e-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/3e9f27bc-2bf5-4b5d-b6b1-d63c5271624e-maas-api-xks-ca\") pod \"3e9f27bc-2bf5-4b5d-b6b1-d63c5271624e\" (UID: \"3e9f27bc-2bf5-4b5d-b6b1-d63c5271624e\") "
Oct 30 22:21:18.220039 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:18.219847    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3e9f27bc-2bf5-4b5d-b6b1-d63c5271624e-kube-api-access-2cv4z\" (UniqueName: \"kubernetes.io/projected/3e9f27bc-2bf5-4b5d-b6b1-d63c5271624e-kube-api-access-2cv4z\") pod \"3e9f27bc-2bf5-4b5d-b6b1-d63c5271624e\" (UID: \"3e9f27bc-2bf5-4b5d-b6b1-d63c5271624e\") "
Oct 30 22:21:18.220039 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:18.219901    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3e9f27bc-2bf5-4b5d-b6b1-d63c5271624e-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/3e9f27bc-2bf5-4b5d-b6b1-d63c5271624e-maas-api-openshift-ca\") pod \"3e9f27bc-2bf5-4b5d-b6b1-d63c5271624e\" (UID: \"3e9f27bc-2bf5-4b5d-b6b1-d63c5271624e\") "
Oct 30 22:21:18.220325 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:18.220296    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3e9f27bc-2bf5-4b5d-b6b1-d63c5271624e-maas-api-openshift-ca" pod "3e9f27bc-2bf5-4b5d-b6b1-d63c5271624e" (UID: "3e9f27bc-2bf5-4b5d-b6b1-d63c5271624e"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:21:18.222111 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:18.222084    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3e9f27bc-2bf5-4b5d-b6b1-d63c5271624e-maas-api-xks-ca" pod "3e9f27bc-2bf5-4b5d-b6b1-d63c5271624e" (UID: "3e9f27bc-2bf5-4b5d-b6b1-d63c5271624e"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:21:18.222195 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:18.222180    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3e9f27bc-2bf5-4b5d-b6b1-d63c5271624e-kube-api-access-2cv4z" pod "3e9f27bc-2bf5-4b5d-b6b1-d63c5271624e" (UID: "3e9f27bc-2bf5-4b5d-b6b1-d63c5271624e"). InnerVolumeSpecName "kube-api-access-2cv4z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:21:18.321076 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:18.320992    4473 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2cv4z\" (UniqueName: \"kubernetes.io/projected/3e9f27bc-2bf5-4b5d-b6b1-d63c5271624e-kube-api-access-2cv4z\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:21:18.321076 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:18.321023    4473 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/3e9f27bc-2bf5-4b5d-b6b1-d63c5271624e-maas-api-openshift-ca\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:21:18.321076 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:18.321035    4473 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/3e9f27bc-2bf5-4b5d-b6b1-d63c5271624e-maas-api-xks-ca\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:21:18.587480 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:18.587391    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w3-0f42cb-566b854747-xhdwq" podUID="17ff1c66-c309-45da-9843-b37262d288d6" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:21:19.000000 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:18.999974    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-459d3dc2-r7c8g"
Oct 30 22:21:19.000199 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:18.999972    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-459d3dc2-r7c8g" event={"ID":"3e9f27bc-2bf5-4b5d-b6b1-d63c5271624e","Type":"ContainerDied","Data":"027c3290ad1622fd2cab9a015eeebe0f29138030c742efe75fc95830a34e76dd"}
Oct 30 22:21:19.000199 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:19.000094    4473 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="027c3290ad1622fd2cab9a015eeebe0f29138030c742efe75fc95830a34e76dd"
Oct 30 22:21:19.224999 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:19.224914    4473 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w3-0f42cb/distinct-2-w3-0f42cb-kserve-fc477897b-jk8qc" podUID="61b49182-057e-41af-b609-1a6abf82c197" containerName="main" containerID="cri-o://15fa7e928582a0d68845747079e81169d32cd3c005241a40730b346d274eb1df" gracePeriod=45
Oct 30 22:21:19.283381 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:19.283297    4473 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w3-0f42cb/distinct-w3-0f42cb-kserve-fb54fdd59-n6w4l" podUID="ff3c2b25-08aa-4b70-bf57-9cf4a11d05ed" containerName="main" containerID="cri-o://0135f7809e015e6c379e57672ed0828bcc1580e15a3b3fc617ec8462060533e4" gracePeriod=45
Oct 30 22:21:19.531648 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:19.531620    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-0f42cb/distinct-2-w3-0f42cb-kserve-fc477897b-jk8qc"
Oct 30 22:21:19.561289 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:19.561265    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-0f42cb/distinct-w3-0f42cb-kserve-fb54fdd59-n6w4l"
Oct 30 22:21:19.634531 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:19.634499    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ff3c2b25-08aa-4b70-bf57-9cf4a11d05ed-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ff3c2b25-08aa-4b70-bf57-9cf4a11d05ed-tmp-dir\") pod \"ff3c2b25-08aa-4b70-bf57-9cf4a11d05ed\" (UID: \"ff3c2b25-08aa-4b70-bf57-9cf4a11d05ed\") "
Oct 30 22:21:19.634531 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:19.634536    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/61b49182-057e-41af-b609-1a6abf82c197-home\" (UniqueName: \"kubernetes.io/empty-dir/61b49182-057e-41af-b609-1a6abf82c197-home\") pod \"61b49182-057e-41af-b609-1a6abf82c197\" (UID: \"61b49182-057e-41af-b609-1a6abf82c197\") "
Oct 30 22:21:19.634785 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:19.634602    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/61b49182-057e-41af-b609-1a6abf82c197-tls-certs\" (UniqueName: \"kubernetes.io/secret/61b49182-057e-41af-b609-1a6abf82c197-tls-certs\") pod \"61b49182-057e-41af-b609-1a6abf82c197\" (UID: \"61b49182-057e-41af-b609-1a6abf82c197\") "
Oct 30 22:21:19.634785 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:19.634648    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/61b49182-057e-41af-b609-1a6abf82c197-kube-api-access-wtn7p\" (UniqueName: \"kubernetes.io/projected/61b49182-057e-41af-b609-1a6abf82c197-kube-api-access-wtn7p\") pod \"61b49182-057e-41af-b609-1a6abf82c197\" (UID: \"61b49182-057e-41af-b609-1a6abf82c197\") "
Oct 30 22:21:19.634785 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:19.634700    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/61b49182-057e-41af-b609-1a6abf82c197-dshm\" (UniqueName: \"kubernetes.io/empty-dir/61b49182-057e-41af-b609-1a6abf82c197-dshm\") pod \"61b49182-057e-41af-b609-1a6abf82c197\" (UID: \"61b49182-057e-41af-b609-1a6abf82c197\") "
Oct 30 22:21:19.634785 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:19.634734    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/61b49182-057e-41af-b609-1a6abf82c197-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/61b49182-057e-41af-b609-1a6abf82c197-tmp-dir\") pod \"61b49182-057e-41af-b609-1a6abf82c197\" (UID: \"61b49182-057e-41af-b609-1a6abf82c197\") "
Oct 30 22:21:19.635036 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:19.634803    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/61b49182-057e-41af-b609-1a6abf82c197-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/61b49182-057e-41af-b609-1a6abf82c197-model-cache\") pod \"61b49182-057e-41af-b609-1a6abf82c197\" (UID: \"61b49182-057e-41af-b609-1a6abf82c197\") "
Oct 30 22:21:19.635036 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:19.634816    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff3c2b25-08aa-4b70-bf57-9cf4a11d05ed-tmp-dir" pod "ff3c2b25-08aa-4b70-bf57-9cf4a11d05ed" (UID: "ff3c2b25-08aa-4b70-bf57-9cf4a11d05ed"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:21:19.635036 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:19.634836    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ff3c2b25-08aa-4b70-bf57-9cf4a11d05ed-home\" (UniqueName: \"kubernetes.io/empty-dir/ff3c2b25-08aa-4b70-bf57-9cf4a11d05ed-home\") pod \"ff3c2b25-08aa-4b70-bf57-9cf4a11d05ed\" (UID: \"ff3c2b25-08aa-4b70-bf57-9cf4a11d05ed\") "
Oct 30 22:21:19.635036 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:19.634863    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/61b49182-057e-41af-b609-1a6abf82c197-home" pod "61b49182-057e-41af-b609-1a6abf82c197" (UID: "61b49182-057e-41af-b609-1a6abf82c197"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:21:19.635036 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:19.634883    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ff3c2b25-08aa-4b70-bf57-9cf4a11d05ed-kube-api-access-7qf9d\" (UniqueName: \"kubernetes.io/projected/ff3c2b25-08aa-4b70-bf57-9cf4a11d05ed-kube-api-access-7qf9d\") pod \"ff3c2b25-08aa-4b70-bf57-9cf4a11d05ed\" (UID: \"ff3c2b25-08aa-4b70-bf57-9cf4a11d05ed\") "
Oct 30 22:21:19.635036 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:19.634956    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ff3c2b25-08aa-4b70-bf57-9cf4a11d05ed-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ff3c2b25-08aa-4b70-bf57-9cf4a11d05ed-model-cache\") pod \"ff3c2b25-08aa-4b70-bf57-9cf4a11d05ed\" (UID: \"ff3c2b25-08aa-4b70-bf57-9cf4a11d05ed\") "
Oct 30 22:21:19.635036 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:19.635012    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ff3c2b25-08aa-4b70-bf57-9cf4a11d05ed-tls-certs\" (UniqueName: \"kubernetes.io/secret/ff3c2b25-08aa-4b70-bf57-9cf4a11d05ed-tls-certs\") pod \"ff3c2b25-08aa-4b70-bf57-9cf4a11d05ed\" (UID: \"ff3c2b25-08aa-4b70-bf57-9cf4a11d05ed\") "
Oct 30 22:21:19.635413 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:19.635042    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ff3c2b25-08aa-4b70-bf57-9cf4a11d05ed-dshm\" (UniqueName: \"kubernetes.io/empty-dir/ff3c2b25-08aa-4b70-bf57-9cf4a11d05ed-dshm\") pod \"ff3c2b25-08aa-4b70-bf57-9cf4a11d05ed\" (UID: \"ff3c2b25-08aa-4b70-bf57-9cf4a11d05ed\") "
Oct 30 22:21:19.635413 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:19.635091    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/61b49182-057e-41af-b609-1a6abf82c197-tmp-dir" pod "61b49182-057e-41af-b609-1a6abf82c197" (UID: "61b49182-057e-41af-b609-1a6abf82c197"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:21:19.635413 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:19.635101    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff3c2b25-08aa-4b70-bf57-9cf4a11d05ed-home" pod "ff3c2b25-08aa-4b70-bf57-9cf4a11d05ed" (UID: "ff3c2b25-08aa-4b70-bf57-9cf4a11d05ed"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:21:19.635413 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:19.635109    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/61b49182-057e-41af-b609-1a6abf82c197-model-cache" pod "61b49182-057e-41af-b609-1a6abf82c197" (UID: "61b49182-057e-41af-b609-1a6abf82c197"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:21:19.635413 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:19.635315    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff3c2b25-08aa-4b70-bf57-9cf4a11d05ed-model-cache" pod "ff3c2b25-08aa-4b70-bf57-9cf4a11d05ed" (UID: "ff3c2b25-08aa-4b70-bf57-9cf4a11d05ed"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:21:19.635678 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:19.635467    4473 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ff3c2b25-08aa-4b70-bf57-9cf4a11d05ed-model-cache\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:21:19.635678 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:19.635489    4473 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ff3c2b25-08aa-4b70-bf57-9cf4a11d05ed-tmp-dir\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:21:19.635678 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:19.635505    4473 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/61b49182-057e-41af-b609-1a6abf82c197-home\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:21:19.635678 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:19.635516    4473 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/61b49182-057e-41af-b609-1a6abf82c197-tmp-dir\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:21:19.635678 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:19.635529    4473 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/61b49182-057e-41af-b609-1a6abf82c197-model-cache\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:21:19.635678 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:19.635541    4473 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ff3c2b25-08aa-4b70-bf57-9cf4a11d05ed-home\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:21:19.637441 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:19.637412    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ff3c2b25-08aa-4b70-bf57-9cf4a11d05ed-kube-api-access-7qf9d" pod "ff3c2b25-08aa-4b70-bf57-9cf4a11d05ed" (UID: "ff3c2b25-08aa-4b70-bf57-9cf4a11d05ed"). InnerVolumeSpecName "kube-api-access-7qf9d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:21:19.637590 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:19.637456    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ff3c2b25-08aa-4b70-bf57-9cf4a11d05ed-tls-certs" pod "ff3c2b25-08aa-4b70-bf57-9cf4a11d05ed" (UID: "ff3c2b25-08aa-4b70-bf57-9cf4a11d05ed"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:21:19.637590 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:19.637536    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/61b49182-057e-41af-b609-1a6abf82c197-dshm" pod "61b49182-057e-41af-b609-1a6abf82c197" (UID: "61b49182-057e-41af-b609-1a6abf82c197"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:21:19.637590 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:19.637557    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/61b49182-057e-41af-b609-1a6abf82c197-kube-api-access-wtn7p" pod "61b49182-057e-41af-b609-1a6abf82c197" (UID: "61b49182-057e-41af-b609-1a6abf82c197"). InnerVolumeSpecName "kube-api-access-wtn7p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:21:19.637845 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:19.637825    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/61b49182-057e-41af-b609-1a6abf82c197-tls-certs" pod "61b49182-057e-41af-b609-1a6abf82c197" (UID: "61b49182-057e-41af-b609-1a6abf82c197"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:21:19.637845 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:19.637832    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff3c2b25-08aa-4b70-bf57-9cf4a11d05ed-dshm" pod "ff3c2b25-08aa-4b70-bf57-9cf4a11d05ed" (UID: "ff3c2b25-08aa-4b70-bf57-9cf4a11d05ed"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:21:19.737209 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:19.737169    4473 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/61b49182-057e-41af-b609-1a6abf82c197-tls-certs\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:21:19.737357 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:19.737214    4473 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wtn7p\" (UniqueName: \"kubernetes.io/projected/61b49182-057e-41af-b609-1a6abf82c197-kube-api-access-wtn7p\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:21:19.737357 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:19.737231    4473 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/61b49182-057e-41af-b609-1a6abf82c197-dshm\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:21:19.737357 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:19.737246    4473 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7qf9d\" (UniqueName: \"kubernetes.io/projected/ff3c2b25-08aa-4b70-bf57-9cf4a11d05ed-kube-api-access-7qf9d\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:21:19.737357 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:19.737261    4473 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ff3c2b25-08aa-4b70-bf57-9cf4a11d05ed-tls-certs\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:21:19.737357 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:19.737274    4473 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ff3c2b25-08aa-4b70-bf57-9cf4a11d05ed-dshm\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:21:20.006237 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:20.006201    4473 generic.go:356] "Generic (PLEG): container finished" podID="ff3c2b25-08aa-4b70-bf57-9cf4a11d05ed" containerID="0135f7809e015e6c379e57672ed0828bcc1580e15a3b3fc617ec8462060533e4" exitCode=0
Oct 30 22:21:20.006458 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:20.006253    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-0f42cb/distinct-w3-0f42cb-kserve-fb54fdd59-n6w4l"
Oct 30 22:21:20.006458 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:20.006283    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-0f42cb/distinct-w3-0f42cb-kserve-fb54fdd59-n6w4l" event={"ID":"ff3c2b25-08aa-4b70-bf57-9cf4a11d05ed","Type":"ContainerDied","Data":"0135f7809e015e6c379e57672ed0828bcc1580e15a3b3fc617ec8462060533e4"}
Oct 30 22:21:20.006458 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:20.006326    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-0f42cb/distinct-w3-0f42cb-kserve-fb54fdd59-n6w4l" event={"ID":"ff3c2b25-08aa-4b70-bf57-9cf4a11d05ed","Type":"ContainerDied","Data":"a2b0ce67416ac69520c8d12bfae61f0b69649d03128ed00a97bc1c0bba7c6b1c"}
Oct 30 22:21:20.006458 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:20.006345    4473 scope.go:122] "RemoveContainer" containerID="0135f7809e015e6c379e57672ed0828bcc1580e15a3b3fc617ec8462060533e4"
Oct 30 22:21:20.007717 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:20.007699    4473 generic.go:356] "Generic (PLEG): container finished" podID="61b49182-057e-41af-b609-1a6abf82c197" containerID="15fa7e928582a0d68845747079e81169d32cd3c005241a40730b346d274eb1df" exitCode=0
Oct 30 22:21:20.007833 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:20.007719    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-0f42cb/distinct-2-w3-0f42cb-kserve-fc477897b-jk8qc" event={"ID":"61b49182-057e-41af-b609-1a6abf82c197","Type":"ContainerDied","Data":"15fa7e928582a0d68845747079e81169d32cd3c005241a40730b346d274eb1df"}
Oct 30 22:21:20.007833 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:20.007747    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-0f42cb/distinct-2-w3-0f42cb-kserve-fc477897b-jk8qc"
Oct 30 22:21:20.007833 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:20.007751    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-0f42cb/distinct-2-w3-0f42cb-kserve-fc477897b-jk8qc" event={"ID":"61b49182-057e-41af-b609-1a6abf82c197","Type":"ContainerDied","Data":"44ebd893326f750559874906f4a3a722bb4739eb8f1223c5b3923b74c58a2c07"}
Oct 30 22:21:20.018030 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:20.018012    4473 scope.go:122] "RemoveContainer" containerID="0135f7809e015e6c379e57672ed0828bcc1580e15a3b3fc617ec8462060533e4"
Oct 30 22:21:20.018353 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:21:20.018334    4473 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0135f7809e015e6c379e57672ed0828bcc1580e15a3b3fc617ec8462060533e4\": container with ID starting with 0135f7809e015e6c379e57672ed0828bcc1580e15a3b3fc617ec8462060533e4 not found: ID does not exist" containerID="0135f7809e015e6c379e57672ed0828bcc1580e15a3b3fc617ec8462060533e4"
Oct 30 22:21:20.018426 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:20.018361    4473 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0135f7809e015e6c379e57672ed0828bcc1580e15a3b3fc617ec8462060533e4"} err="failed to get container status \"0135f7809e015e6c379e57672ed0828bcc1580e15a3b3fc617ec8462060533e4\": rpc error: code = NotFound desc = could not find container \"0135f7809e015e6c379e57672ed0828bcc1580e15a3b3fc617ec8462060533e4\": container with ID starting with 0135f7809e015e6c379e57672ed0828bcc1580e15a3b3fc617ec8462060533e4 not found: ID does not exist"
Oct 30 22:21:20.018426 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:20.018381    4473 scope.go:122] "RemoveContainer" containerID="15fa7e928582a0d68845747079e81169d32cd3c005241a40730b346d274eb1df"
Oct 30 22:21:20.027044 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:20.027026    4473 scope.go:122] "RemoveContainer" containerID="15fa7e928582a0d68845747079e81169d32cd3c005241a40730b346d274eb1df"
Oct 30 22:21:20.027344 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:21:20.027328    4473 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"15fa7e928582a0d68845747079e81169d32cd3c005241a40730b346d274eb1df\": container with ID starting with 15fa7e928582a0d68845747079e81169d32cd3c005241a40730b346d274eb1df not found: ID does not exist" containerID="15fa7e928582a0d68845747079e81169d32cd3c005241a40730b346d274eb1df"
Oct 30 22:21:20.027387 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:20.027353    4473 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"15fa7e928582a0d68845747079e81169d32cd3c005241a40730b346d274eb1df"} err="failed to get container status \"15fa7e928582a0d68845747079e81169d32cd3c005241a40730b346d274eb1df\": rpc error: code = NotFound desc = could not find container \"15fa7e928582a0d68845747079e81169d32cd3c005241a40730b346d274eb1df\": container with ID starting with 15fa7e928582a0d68845747079e81169d32cd3c005241a40730b346d274eb1df not found: ID does not exist"
Oct 30 22:21:20.173235 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:20.173206    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-0f42cb/distinct-w3-0f42cb-kserve-fb54fdd59-n6w4l"]
Oct 30 22:21:20.175013 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:20.174992    4473 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w3-0f42cb/distinct-w3-0f42cb-kserve-fb54fdd59-n6w4l"]
Oct 30 22:21:20.298333 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:20.298250    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-0f42cb/distinct-2-w3-0f42cb-kserve-fc477897b-jk8qc"]
Oct 30 22:21:20.312179 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:20.312151    4473 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w3-0f42cb/distinct-2-w3-0f42cb-kserve-fc477897b-jk8qc"]
Oct 30 22:21:20.324407 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:21:20.324372    4473 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fncvg" containerName="extractor"
Oct 30 22:21:20.356936 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:20.356898    4473 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="61b49182-057e-41af-b609-1a6abf82c197" path="/var/lib/kubelet/pods/61b49182-057e-41af-b609-1a6abf82c197/volumes"
Oct 30 22:21:20.357353 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:20.357339    4473 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ff3c2b25-08aa-4b70-bf57-9cf4a11d05ed" path="/var/lib/kubelet/pods/ff3c2b25-08aa-4b70-bf57-9cf4a11d05ed/volumes"
Oct 30 22:21:28.586762 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:28.586724    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w3-0f42cb-566b854747-xhdwq" podUID="17ff1c66-c309-45da-9843-b37262d288d6" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:21:36.338817 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:36.338790    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w3-0f42cb-566b854747-xhdwq"
Oct 30 22:21:36.417411 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:36.417379    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/17ff1c66-c309-45da-9843-b37262d288d6-kube-api-access-vfqj4\" (UniqueName: \"kubernetes.io/projected/17ff1c66-c309-45da-9843-b37262d288d6-kube-api-access-vfqj4\") pod \"17ff1c66-c309-45da-9843-b37262d288d6\" (UID: \"17ff1c66-c309-45da-9843-b37262d288d6\") "
Oct 30 22:21:36.417604 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:36.417437    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/17ff1c66-c309-45da-9843-b37262d288d6-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/17ff1c66-c309-45da-9843-b37262d288d6-plugins-config-volume\") pod \"17ff1c66-c309-45da-9843-b37262d288d6\" (UID: \"17ff1c66-c309-45da-9843-b37262d288d6\") "
Oct 30 22:21:36.417865 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:36.417840    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/17ff1c66-c309-45da-9843-b37262d288d6-plugins-config-volume" pod "17ff1c66-c309-45da-9843-b37262d288d6" (UID: "17ff1c66-c309-45da-9843-b37262d288d6"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:21:36.419767 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:36.419737    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/17ff1c66-c309-45da-9843-b37262d288d6-kube-api-access-vfqj4" pod "17ff1c66-c309-45da-9843-b37262d288d6" (UID: "17ff1c66-c309-45da-9843-b37262d288d6"). InnerVolumeSpecName "kube-api-access-vfqj4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:21:36.518685 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:36.518651    4473 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vfqj4\" (UniqueName: \"kubernetes.io/projected/17ff1c66-c309-45da-9843-b37262d288d6-kube-api-access-vfqj4\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:21:36.518685 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:36.518683    4473 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/17ff1c66-c309-45da-9843-b37262d288d6-plugins-config-volume\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:21:37.091954 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:37.091901    4473 generic.go:356] "Generic (PLEG): container finished" podID="17ff1c66-c309-45da-9843-b37262d288d6" containerID="c47c4349be6968d4cac79d5fd1241de583baae04f584f112f57c91281fca9054" exitCode=0
Oct 30 22:21:37.092170 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:37.091974    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w3-0f42cb-566b854747-xhdwq"
Oct 30 22:21:37.092170 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:37.091984    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w3-0f42cb-566b854747-xhdwq" event={"ID":"17ff1c66-c309-45da-9843-b37262d288d6","Type":"ContainerDied","Data":"c47c4349be6968d4cac79d5fd1241de583baae04f584f112f57c91281fca9054"}
Oct 30 22:21:37.092170 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:37.092040    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w3-0f42cb-566b854747-xhdwq" event={"ID":"17ff1c66-c309-45da-9843-b37262d288d6","Type":"ContainerDied","Data":"8ad9d83d64fb3954c41ed8a89bf61a36a33cd3e3baa2d1b748f0157da6b7d713"}
Oct 30 22:21:37.092170 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:37.092063    4473 scope.go:122] "RemoveContainer" containerID="c47c4349be6968d4cac79d5fd1241de583baae04f584f112f57c91281fca9054"
Oct 30 22:21:37.101377 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:37.101359    4473 scope.go:122] "RemoveContainer" containerID="c47c4349be6968d4cac79d5fd1241de583baae04f584f112f57c91281fca9054"
Oct 30 22:21:37.101680 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:21:37.101660    4473 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c47c4349be6968d4cac79d5fd1241de583baae04f584f112f57c91281fca9054\": container with ID starting with c47c4349be6968d4cac79d5fd1241de583baae04f584f112f57c91281fca9054 not found: ID does not exist" containerID="c47c4349be6968d4cac79d5fd1241de583baae04f584f112f57c91281fca9054"
Oct 30 22:21:37.101719 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:37.101691    4473 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c47c4349be6968d4cac79d5fd1241de583baae04f584f112f57c91281fca9054"} err="failed to get container status \"c47c4349be6968d4cac79d5fd1241de583baae04f584f112f57c91281fca9054\": rpc error: code = NotFound desc = could not find container \"c47c4349be6968d4cac79d5fd1241de583baae04f584f112f57c91281fca9054\": container with ID starting with c47c4349be6968d4cac79d5fd1241de583baae04f584f112f57c91281fca9054 not found: ID does not exist"
Oct 30 22:21:37.259442 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:37.259408    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-0f42cb-566b854747-xhdwq"]
Oct 30 22:21:37.309970 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:37.309938    4473 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-0f42cb-566b854747-xhdwq"]
Oct 30 22:21:38.355254 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:38.355221    4473 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="17ff1c66-c309-45da-9843-b37262d288d6" path="/var/lib/kubelet/pods/17ff1c66-c309-45da-9843-b37262d288d6/volumes"
Oct 30 22:21:44.809745 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:44.809713    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-786df48447-m2d88"]
Oct 30 22:21:44.810180 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:44.809911    4473 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/maas-controller-786df48447-m2d88" podUID="e5ce0eb3-a29d-4c95-b72f-a84a43f98ea9" containerName="manager" containerID="cri-o://17b0eb34ebff9edb126f961c4de1126619cd3675f50d16ecd274e447fe7b0410" gracePeriod=30
Oct 30 22:21:45.083363 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:45.083340    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-786df48447-m2d88"
Oct 30 22:21:45.136087 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:45.136038    4473 generic.go:356] "Generic (PLEG): container finished" podID="e5ce0eb3-a29d-4c95-b72f-a84a43f98ea9" containerID="17b0eb34ebff9edb126f961c4de1126619cd3675f50d16ecd274e447fe7b0410" exitCode=0
Oct 30 22:21:45.136087 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:45.136082    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-786df48447-m2d88"
Oct 30 22:21:45.136354 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:45.136119    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-786df48447-m2d88" event={"ID":"e5ce0eb3-a29d-4c95-b72f-a84a43f98ea9","Type":"ContainerDied","Data":"17b0eb34ebff9edb126f961c4de1126619cd3675f50d16ecd274e447fe7b0410"}
Oct 30 22:21:45.136354 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:45.136172    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-786df48447-m2d88" event={"ID":"e5ce0eb3-a29d-4c95-b72f-a84a43f98ea9","Type":"ContainerDied","Data":"ba1078d24136e40ae6f639cebe784c9b0a90384cadb4440246fdfaa5cdd8d33d"}
Oct 30 22:21:45.136354 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:45.136199    4473 scope.go:122] "RemoveContainer" containerID="17b0eb34ebff9edb126f961c4de1126619cd3675f50d16ecd274e447fe7b0410"
Oct 30 22:21:45.145256 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:45.145237    4473 scope.go:122] "RemoveContainer" containerID="17b0eb34ebff9edb126f961c4de1126619cd3675f50d16ecd274e447fe7b0410"
Oct 30 22:21:45.145541 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:21:45.145522    4473 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"17b0eb34ebff9edb126f961c4de1126619cd3675f50d16ecd274e447fe7b0410\": container with ID starting with 17b0eb34ebff9edb126f961c4de1126619cd3675f50d16ecd274e447fe7b0410 not found: ID does not exist" containerID="17b0eb34ebff9edb126f961c4de1126619cd3675f50d16ecd274e447fe7b0410"
Oct 30 22:21:45.145586 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:45.145552    4473 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"17b0eb34ebff9edb126f961c4de1126619cd3675f50d16ecd274e447fe7b0410"} err="failed to get container status \"17b0eb34ebff9edb126f961c4de1126619cd3675f50d16ecd274e447fe7b0410\": rpc error: code = NotFound desc = could not find container \"17b0eb34ebff9edb126f961c4de1126619cd3675f50d16ecd274e447fe7b0410\": container with ID starting with 17b0eb34ebff9edb126f961c4de1126619cd3675f50d16ecd274e447fe7b0410 not found: ID does not exist"
Oct 30 22:21:45.210125 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:45.210084    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e5ce0eb3-a29d-4c95-b72f-a84a43f98ea9-metrics-certs\" (UniqueName: \"kubernetes.io/secret/e5ce0eb3-a29d-4c95-b72f-a84a43f98ea9-metrics-certs\") pod \"e5ce0eb3-a29d-4c95-b72f-a84a43f98ea9\" (UID: \"e5ce0eb3-a29d-4c95-b72f-a84a43f98ea9\") "
Oct 30 22:21:45.210125 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:45.210127    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e5ce0eb3-a29d-4c95-b72f-a84a43f98ea9-webhook-cert\" (UniqueName: \"kubernetes.io/secret/e5ce0eb3-a29d-4c95-b72f-a84a43f98ea9-webhook-cert\") pod \"e5ce0eb3-a29d-4c95-b72f-a84a43f98ea9\" (UID: \"e5ce0eb3-a29d-4c95-b72f-a84a43f98ea9\") "
Oct 30 22:21:45.210338 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:45.210229    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e5ce0eb3-a29d-4c95-b72f-a84a43f98ea9-kube-api-access-cx8p4\" (UniqueName: \"kubernetes.io/projected/e5ce0eb3-a29d-4c95-b72f-a84a43f98ea9-kube-api-access-cx8p4\") pod \"e5ce0eb3-a29d-4c95-b72f-a84a43f98ea9\" (UID: \"e5ce0eb3-a29d-4c95-b72f-a84a43f98ea9\") "
Oct 30 22:21:45.212834 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:45.212800    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e5ce0eb3-a29d-4c95-b72f-a84a43f98ea9-webhook-cert" pod "e5ce0eb3-a29d-4c95-b72f-a84a43f98ea9" (UID: "e5ce0eb3-a29d-4c95-b72f-a84a43f98ea9"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:21:45.213435 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:45.213384    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e5ce0eb3-a29d-4c95-b72f-a84a43f98ea9-kube-api-access-cx8p4" pod "e5ce0eb3-a29d-4c95-b72f-a84a43f98ea9" (UID: "e5ce0eb3-a29d-4c95-b72f-a84a43f98ea9"). InnerVolumeSpecName "kube-api-access-cx8p4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:21:45.216276 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:45.216251    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e5ce0eb3-a29d-4c95-b72f-a84a43f98ea9-metrics-certs" pod "e5ce0eb3-a29d-4c95-b72f-a84a43f98ea9" (UID: "e5ce0eb3-a29d-4c95-b72f-a84a43f98ea9"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:21:45.311843 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:45.311760    4473 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cx8p4\" (UniqueName: \"kubernetes.io/projected/e5ce0eb3-a29d-4c95-b72f-a84a43f98ea9-kube-api-access-cx8p4\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:21:45.311843 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:45.311791    4473 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e5ce0eb3-a29d-4c95-b72f-a84a43f98ea9-metrics-certs\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:21:45.311843 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:45.311804    4473 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e5ce0eb3-a29d-4c95-b72f-a84a43f98ea9-webhook-cert\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:21:45.558242 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:45.558203    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-786df48447-m2d88"]
Oct 30 22:21:45.562819 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:45.562736    4473 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-786df48447-m2d88"]
Oct 30 22:21:46.355999 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:46.355959    4473 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e5ce0eb3-a29d-4c95-b72f-a84a43f98ea9" path="/var/lib/kubelet/pods/e5ce0eb3-a29d-4c95-b72f-a84a43f98ea9/volumes"
Oct 30 22:21:46.426598 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:46.426568    4473 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-786df48447-n8fxs"]
Oct 30 22:21:46.427018 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:46.426999    4473 cpu_manager.go:414] "containerMap: removing container" podUID="61b49182-057e-41af-b609-1a6abf82c197" containerName="main"
Oct 30 22:21:46.427110 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:46.427019    4473 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="61b49182-057e-41af-b609-1a6abf82c197" containerName="main"
Oct 30 22:21:46.427110 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:46.427061    4473 cpu_manager.go:414] "containerMap: removing container" podUID="e5ce0eb3-a29d-4c95-b72f-a84a43f98ea9" containerName="manager"
Oct 30 22:21:46.427110 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:46.427067    4473 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e5ce0eb3-a29d-4c95-b72f-a84a43f98ea9" containerName="manager"
Oct 30 22:21:46.427110 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:46.427084    4473 cpu_manager.go:414] "containerMap: removing container" podUID="3e9f27bc-2bf5-4b5d-b6b1-d63c5271624e" containerName="revoke-keys"
Oct 30 22:21:46.427110 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:46.427088    4473 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3e9f27bc-2bf5-4b5d-b6b1-d63c5271624e" containerName="revoke-keys"
Oct 30 22:21:46.427110 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:46.427095    4473 cpu_manager.go:414] "containerMap: removing container" podUID="ff3c2b25-08aa-4b70-bf57-9cf4a11d05ed" containerName="main"
Oct 30 22:21:46.427110 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:46.427099    4473 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ff3c2b25-08aa-4b70-bf57-9cf4a11d05ed" containerName="main"
Oct 30 22:21:46.427110 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:46.427108    4473 cpu_manager.go:414] "containerMap: removing container" podUID="17ff1c66-c309-45da-9843-b37262d288d6" containerName="payload-processing"
Oct 30 22:21:46.427110 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:46.427113    4473 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="17ff1c66-c309-45da-9843-b37262d288d6" containerName="payload-processing"
Oct 30 22:21:46.427579 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:46.427224    4473 memory_manager.go:365] "RemoveStaleState removing state" podUID="17ff1c66-c309-45da-9843-b37262d288d6" containerName="payload-processing"
Oct 30 22:21:46.427579 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:46.427242    4473 memory_manager.go:365] "RemoveStaleState removing state" podUID="61b49182-057e-41af-b609-1a6abf82c197" containerName="main"
Oct 30 22:21:46.427579 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:46.427256    4473 memory_manager.go:365] "RemoveStaleState removing state" podUID="e5ce0eb3-a29d-4c95-b72f-a84a43f98ea9" containerName="manager"
Oct 30 22:21:46.427579 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:46.427269    4473 memory_manager.go:365] "RemoveStaleState removing state" podUID="3e9f27bc-2bf5-4b5d-b6b1-d63c5271624e" containerName="revoke-keys"
Oct 30 22:21:46.427579 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:46.427277    4473 memory_manager.go:365] "RemoveStaleState removing state" podUID="ff3c2b25-08aa-4b70-bf57-9cf4a11d05ed" containerName="main"
Oct 30 22:21:46.431799 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:46.431781    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-786df48447-n8fxs"
Oct 30 22:21:46.487224 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:46.487184    4473 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Oct 30 22:21:46.488795 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:46.488764    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-786df48447-n8fxs"]
Oct 30 22:21:46.524238 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:46.524205    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/16b44c9a-84e3-48a9-983f-2bb08d0bbb9e-webhook-cert\") pod \"maas-controller-786df48447-n8fxs\" (UID: \"16b44c9a-84e3-48a9-983f-2bb08d0bbb9e\") " pod="opendatahub/maas-controller-786df48447-n8fxs"
Oct 30 22:21:46.524426 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:46.524378    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qwz7c\" (UniqueName: \"kubernetes.io/projected/16b44c9a-84e3-48a9-983f-2bb08d0bbb9e-kube-api-access-qwz7c\") pod \"maas-controller-786df48447-n8fxs\" (UID: \"16b44c9a-84e3-48a9-983f-2bb08d0bbb9e\") " pod="opendatahub/maas-controller-786df48447-n8fxs"
Oct 30 22:21:46.524484 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:46.524472    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/16b44c9a-84e3-48a9-983f-2bb08d0bbb9e-metrics-certs\") pod \"maas-controller-786df48447-n8fxs\" (UID: \"16b44c9a-84e3-48a9-983f-2bb08d0bbb9e\") " pod="opendatahub/maas-controller-786df48447-n8fxs"
Oct 30 22:21:46.536507 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:46.536476    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Oct 30 22:21:46.536647 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:46.536513    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Oct 30 22:21:46.536647 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:46.536558    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-qcnnj\""
Oct 30 22:21:46.625613 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:46.625583    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qwz7c\" (UniqueName: \"kubernetes.io/projected/16b44c9a-84e3-48a9-983f-2bb08d0bbb9e-kube-api-access-qwz7c\") pod \"maas-controller-786df48447-n8fxs\" (UID: \"16b44c9a-84e3-48a9-983f-2bb08d0bbb9e\") " pod="opendatahub/maas-controller-786df48447-n8fxs"
Oct 30 22:21:46.625805 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:46.625644    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/16b44c9a-84e3-48a9-983f-2bb08d0bbb9e-metrics-certs\") pod \"maas-controller-786df48447-n8fxs\" (UID: \"16b44c9a-84e3-48a9-983f-2bb08d0bbb9e\") " pod="opendatahub/maas-controller-786df48447-n8fxs"
Oct 30 22:21:46.625805 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:46.625675    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/16b44c9a-84e3-48a9-983f-2bb08d0bbb9e-webhook-cert\") pod \"maas-controller-786df48447-n8fxs\" (UID: \"16b44c9a-84e3-48a9-983f-2bb08d0bbb9e\") " pod="opendatahub/maas-controller-786df48447-n8fxs"
Oct 30 22:21:46.628365 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:46.628337    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/16b44c9a-84e3-48a9-983f-2bb08d0bbb9e-webhook-cert\") pod \"maas-controller-786df48447-n8fxs\" (UID: \"16b44c9a-84e3-48a9-983f-2bb08d0bbb9e\") " pod="opendatahub/maas-controller-786df48447-n8fxs"
Oct 30 22:21:46.628480 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:46.628463    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/16b44c9a-84e3-48a9-983f-2bb08d0bbb9e-metrics-certs\") pod \"maas-controller-786df48447-n8fxs\" (UID: \"16b44c9a-84e3-48a9-983f-2bb08d0bbb9e\") " pod="opendatahub/maas-controller-786df48447-n8fxs"
Oct 30 22:21:46.683729 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:46.683697    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qwz7c\" (UniqueName: \"kubernetes.io/projected/16b44c9a-84e3-48a9-983f-2bb08d0bbb9e-kube-api-access-qwz7c\") pod \"maas-controller-786df48447-n8fxs\" (UID: \"16b44c9a-84e3-48a9-983f-2bb08d0bbb9e\") " pod="opendatahub/maas-controller-786df48447-n8fxs"
Oct 30 22:21:46.744117 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:46.744079    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-786df48447-n8fxs"
Oct 30 22:21:47.115337 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:47.115311    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-786df48447-n8fxs"]
Oct 30 22:21:47.149815 ip-10-0-140-18 kubenswrapper[4473]: W1030 22:21:47.149788    4473 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod16b44c9a_84e3_48a9_983f_2bb08d0bbb9e.slice/crio-0f5cef488c41b6452ec0568b98caf58dd4f57711061a64e8eb68e83917ff9106 WatchSource:0}: Error finding container 0f5cef488c41b6452ec0568b98caf58dd4f57711061a64e8eb68e83917ff9106: Status 404 returned error can't find the container with id 0f5cef488c41b6452ec0568b98caf58dd4f57711061a64e8eb68e83917ff9106
Oct 30 22:21:48.155956 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:48.155873    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-786df48447-n8fxs" event={"ID":"16b44c9a-84e3-48a9-983f-2bb08d0bbb9e","Type":"ContainerStarted","Data":"3fcb9fe2f2a5a9537aab3c5e8fbf46ce73c839abaa415d1336cc066d9c36ffa1"}
Oct 30 22:21:48.156517 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:48.155966    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-786df48447-n8fxs" event={"ID":"16b44c9a-84e3-48a9-983f-2bb08d0bbb9e","Type":"ContainerStarted","Data":"0f5cef488c41b6452ec0568b98caf58dd4f57711061a64e8eb68e83917ff9106"}
Oct 30 22:21:48.156862 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:21:48.156839    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-786df48447-n8fxs"
Oct 30 22:21:50.324366 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:21:50.324327    4473 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fncvg" containerName="extractor"
Oct 30 22:22:00.180733 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:22:00.180700    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-786df48447-n8fxs"
Oct 30 22:22:00.336576 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:22:00.336510    4473 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/maas-controller-786df48447-n8fxs" podStartSLOduration=13.470484730999999 podStartE2EDuration="14.336493759s" totalImagesPullingTime="866.009028ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:21:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:21:48.339410493 +0000 UTC m=+2354.112778981" watchObservedRunningTime="2026-10-30 22:22:00.336493759 +0000 UTC m=+2366.109862287"
Oct 30 22:22:19.169245 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:22:19.169157    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-786df48447-n8fxs"]
Oct 30 22:22:19.170004 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:22:19.169372    4473 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/maas-controller-786df48447-n8fxs" podUID="16b44c9a-84e3-48a9-983f-2bb08d0bbb9e" containerName="manager" containerID="cri-o://3fcb9fe2f2a5a9537aab3c5e8fbf46ce73c839abaa415d1336cc066d9c36ffa1" gracePeriod=30
Oct 30 22:22:19.325954 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:22:19.325887    4473 generic.go:356] "Generic (PLEG): container finished" podID="16b44c9a-84e3-48a9-983f-2bb08d0bbb9e" containerID="3fcb9fe2f2a5a9537aab3c5e8fbf46ce73c839abaa415d1336cc066d9c36ffa1" exitCode=0
Oct 30 22:22:19.326298 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:22:19.325962    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-786df48447-n8fxs" event={"ID":"16b44c9a-84e3-48a9-983f-2bb08d0bbb9e","Type":"ContainerDied","Data":"3fcb9fe2f2a5a9537aab3c5e8fbf46ce73c839abaa415d1336cc066d9c36ffa1"}
Oct 30 22:22:19.427895 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:22:19.427831    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-786df48447-n8fxs"
Oct 30 22:22:19.490388 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:22:19.490348    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/16b44c9a-84e3-48a9-983f-2bb08d0bbb9e-webhook-cert\" (UniqueName: \"kubernetes.io/secret/16b44c9a-84e3-48a9-983f-2bb08d0bbb9e-webhook-cert\") pod \"16b44c9a-84e3-48a9-983f-2bb08d0bbb9e\" (UID: \"16b44c9a-84e3-48a9-983f-2bb08d0bbb9e\") "
Oct 30 22:22:19.490584 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:22:19.490443    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/16b44c9a-84e3-48a9-983f-2bb08d0bbb9e-metrics-certs\" (UniqueName: \"kubernetes.io/secret/16b44c9a-84e3-48a9-983f-2bb08d0bbb9e-metrics-certs\") pod \"16b44c9a-84e3-48a9-983f-2bb08d0bbb9e\" (UID: \"16b44c9a-84e3-48a9-983f-2bb08d0bbb9e\") "
Oct 30 22:22:19.490584 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:22:19.490484    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/16b44c9a-84e3-48a9-983f-2bb08d0bbb9e-kube-api-access-qwz7c\" (UniqueName: \"kubernetes.io/projected/16b44c9a-84e3-48a9-983f-2bb08d0bbb9e-kube-api-access-qwz7c\") pod \"16b44c9a-84e3-48a9-983f-2bb08d0bbb9e\" (UID: \"16b44c9a-84e3-48a9-983f-2bb08d0bbb9e\") "
Oct 30 22:22:19.492788 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:22:19.492759    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/16b44c9a-84e3-48a9-983f-2bb08d0bbb9e-metrics-certs" pod "16b44c9a-84e3-48a9-983f-2bb08d0bbb9e" (UID: "16b44c9a-84e3-48a9-983f-2bb08d0bbb9e"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:22:19.492911 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:22:19.492788    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/16b44c9a-84e3-48a9-983f-2bb08d0bbb9e-webhook-cert" pod "16b44c9a-84e3-48a9-983f-2bb08d0bbb9e" (UID: "16b44c9a-84e3-48a9-983f-2bb08d0bbb9e"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:22:19.492911 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:22:19.492795    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/16b44c9a-84e3-48a9-983f-2bb08d0bbb9e-kube-api-access-qwz7c" pod "16b44c9a-84e3-48a9-983f-2bb08d0bbb9e" (UID: "16b44c9a-84e3-48a9-983f-2bb08d0bbb9e"). InnerVolumeSpecName "kube-api-access-qwz7c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:22:19.591938 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:22:19.591894    4473 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/16b44c9a-84e3-48a9-983f-2bb08d0bbb9e-webhook-cert\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:22:19.591938 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:22:19.591939    4473 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/16b44c9a-84e3-48a9-983f-2bb08d0bbb9e-metrics-certs\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:22:19.592147 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:22:19.591952    4473 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qwz7c\" (UniqueName: \"kubernetes.io/projected/16b44c9a-84e3-48a9-983f-2bb08d0bbb9e-kube-api-access-qwz7c\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:22:20.323823 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:22:20.323779    4473 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fncvg" containerName="extractor"
Oct 30 22:22:20.332961 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:22:20.332930    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-786df48447-n8fxs"
Oct 30 22:22:20.332961 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:22:20.332944    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-786df48447-n8fxs" event={"ID":"16b44c9a-84e3-48a9-983f-2bb08d0bbb9e","Type":"ContainerDied","Data":"0f5cef488c41b6452ec0568b98caf58dd4f57711061a64e8eb68e83917ff9106"}
Oct 30 22:22:20.333197 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:22:20.333006    4473 scope.go:122] "RemoveContainer" containerID="3fcb9fe2f2a5a9537aab3c5e8fbf46ce73c839abaa415d1336cc066d9c36ffa1"
Oct 30 22:22:20.498339 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:22:20.498301    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-786df48447-n8fxs"]
Oct 30 22:22:20.548894 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:22:20.548860    4473 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-786df48447-n8fxs"]
Oct 30 22:22:21.818573 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:22:21.818534    4473 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-786df48447-pn8gm"]
Oct 30 22:22:21.819003 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:22:21.818974    4473 cpu_manager.go:414] "containerMap: removing container" podUID="16b44c9a-84e3-48a9-983f-2bb08d0bbb9e" containerName="manager"
Oct 30 22:22:21.819003 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:22:21.818986    4473 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="16b44c9a-84e3-48a9-983f-2bb08d0bbb9e" containerName="manager"
Oct 30 22:22:21.819088 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:22:21.819079    4473 memory_manager.go:365] "RemoveStaleState removing state" podUID="16b44c9a-84e3-48a9-983f-2bb08d0bbb9e" containerName="manager"
Oct 30 22:22:21.823606 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:22:21.823584    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-786df48447-pn8gm"
Oct 30 22:22:21.876683 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:22:21.876657    4473 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Oct 30 22:22:21.916476 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:22:21.916436    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6c7988f6-b9bc-4009-8354-73efd639c648-webhook-cert\") pod \"maas-controller-786df48447-pn8gm\" (UID: \"6c7988f6-b9bc-4009-8354-73efd639c648\") " pod="opendatahub/maas-controller-786df48447-pn8gm"
Oct 30 22:22:21.916662 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:22:21.916486    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qbp2m\" (UniqueName: \"kubernetes.io/projected/6c7988f6-b9bc-4009-8354-73efd639c648-kube-api-access-qbp2m\") pod \"maas-controller-786df48447-pn8gm\" (UID: \"6c7988f6-b9bc-4009-8354-73efd639c648\") " pod="opendatahub/maas-controller-786df48447-pn8gm"
Oct 30 22:22:21.916662 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:22:21.916562    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6c7988f6-b9bc-4009-8354-73efd639c648-metrics-certs\") pod \"maas-controller-786df48447-pn8gm\" (UID: \"6c7988f6-b9bc-4009-8354-73efd639c648\") " pod="opendatahub/maas-controller-786df48447-pn8gm"
Oct 30 22:22:21.927184 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:22:21.927154    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-qcnnj\""
Oct 30 22:22:21.927184 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:22:21.927156    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Oct 30 22:22:21.927420 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:22:21.927315    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Oct 30 22:22:21.930360 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:22:21.930331    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-786df48447-pn8gm"]
Oct 30 22:22:22.017996 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:22:22.017943    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6c7988f6-b9bc-4009-8354-73efd639c648-webhook-cert\") pod \"maas-controller-786df48447-pn8gm\" (UID: \"6c7988f6-b9bc-4009-8354-73efd639c648\") " pod="opendatahub/maas-controller-786df48447-pn8gm"
Oct 30 22:22:22.018224 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:22:22.018018    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qbp2m\" (UniqueName: \"kubernetes.io/projected/6c7988f6-b9bc-4009-8354-73efd639c648-kube-api-access-qbp2m\") pod \"maas-controller-786df48447-pn8gm\" (UID: \"6c7988f6-b9bc-4009-8354-73efd639c648\") " pod="opendatahub/maas-controller-786df48447-pn8gm"
Oct 30 22:22:22.018224 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:22:22.018047    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6c7988f6-b9bc-4009-8354-73efd639c648-metrics-certs\") pod \"maas-controller-786df48447-pn8gm\" (UID: \"6c7988f6-b9bc-4009-8354-73efd639c648\") " pod="opendatahub/maas-controller-786df48447-pn8gm"
Oct 30 22:22:22.020600 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:22:22.020574    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6c7988f6-b9bc-4009-8354-73efd639c648-metrics-certs\") pod \"maas-controller-786df48447-pn8gm\" (UID: \"6c7988f6-b9bc-4009-8354-73efd639c648\") " pod="opendatahub/maas-controller-786df48447-pn8gm"
Oct 30 22:22:22.020715 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:22:22.020687    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6c7988f6-b9bc-4009-8354-73efd639c648-webhook-cert\") pod \"maas-controller-786df48447-pn8gm\" (UID: \"6c7988f6-b9bc-4009-8354-73efd639c648\") " pod="opendatahub/maas-controller-786df48447-pn8gm"
Oct 30 22:22:22.076797 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:22:22.076721    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qbp2m\" (UniqueName: \"kubernetes.io/projected/6c7988f6-b9bc-4009-8354-73efd639c648-kube-api-access-qbp2m\") pod \"maas-controller-786df48447-pn8gm\" (UID: \"6c7988f6-b9bc-4009-8354-73efd639c648\") " pod="opendatahub/maas-controller-786df48447-pn8gm"
Oct 30 22:22:22.137510 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:22:22.137473    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-786df48447-pn8gm"
Oct 30 22:22:22.355542 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:22:22.355460    4473 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="16b44c9a-84e3-48a9-983f-2bb08d0bbb9e" path="/var/lib/kubelet/pods/16b44c9a-84e3-48a9-983f-2bb08d0bbb9e/volumes"
Oct 30 22:22:22.512773 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:22:22.512743    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-786df48447-pn8gm"]
Oct 30 22:22:22.514008 ip-10-0-140-18 kubenswrapper[4473]: W1030 22:22:22.513981    4473 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c7988f6_b9bc_4009_8354_73efd639c648.slice/crio-04af58ca7da030b2a28d641dec1aabf32a5ea4b467cc628c6f239018c592c86a WatchSource:0}: Error finding container 04af58ca7da030b2a28d641dec1aabf32a5ea4b467cc628c6f239018c592c86a: Status 404 returned error can't find the container with id 04af58ca7da030b2a28d641dec1aabf32a5ea4b467cc628c6f239018c592c86a
Oct 30 22:22:23.350445 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:22:23.350421    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-786df48447-pn8gm" event={"ID":"6c7988f6-b9bc-4009-8354-73efd639c648","Type":"ContainerStarted","Data":"04af58ca7da030b2a28d641dec1aabf32a5ea4b467cc628c6f239018c592c86a"}
Oct 30 22:22:24.357224 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:22:24.357183    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-786df48447-pn8gm" event={"ID":"6c7988f6-b9bc-4009-8354-73efd639c648","Type":"ContainerStarted","Data":"d37b000e25ebfe8f08e4224843cd1f99993677f00736a9085dcda843ed2366f6"}
Oct 30 22:22:24.357705 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:22:24.357244    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-786df48447-pn8gm"
Oct 30 22:22:24.476689 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:22:24.476633    4473 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/maas-controller-786df48447-pn8gm" podStartSLOduration=2.697229622 podStartE2EDuration="3.476616259s" totalImagesPullingTime="779.386637ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:22:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:22:24.475815326 +0000 UTC m=+2390.249183816" watchObservedRunningTime="2026-10-30 22:22:24.476616259 +0000 UTC m=+2390.249984737"
Oct 30 22:22:35.365595 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:22:35.365565    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-786df48447-pn8gm"
Oct 30 22:22:50.150279 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:22:50.150242    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-lxkmp"]
Oct 30 22:22:50.150856 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:22:50.150441    4473 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-lxkmp" podUID="8b90548f-3da8-42df-99d0-d5eb415a4af3" containerName="discovery" containerID="cri-o://153757e7bbcb55858fe522c63c89b4d27454da55a5d79eafc8df23c63b75eff3" gracePeriod=30
Oct 30 22:22:50.326404 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:22:50.326068    4473 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fncvg" containerName="extractor"
Oct 30 22:22:50.411263 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:22:50.411200    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-lxkmp"
Oct 30 22:22:50.488691 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:22:50.488650    4473 generic.go:356] "Generic (PLEG): container finished" podID="8b90548f-3da8-42df-99d0-d5eb415a4af3" containerID="153757e7bbcb55858fe522c63c89b4d27454da55a5d79eafc8df23c63b75eff3" exitCode=0
Oct 30 22:22:50.488885 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:22:50.488696    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-lxkmp"
Oct 30 22:22:50.488885 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:22:50.488717    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-lxkmp" event={"ID":"8b90548f-3da8-42df-99d0-d5eb415a4af3","Type":"ContainerDied","Data":"153757e7bbcb55858fe522c63c89b4d27454da55a5d79eafc8df23c63b75eff3"}
Oct 30 22:22:50.488885 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:22:50.488778    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-lxkmp" event={"ID":"8b90548f-3da8-42df-99d0-d5eb415a4af3","Type":"ContainerDied","Data":"cb8eaf86028a3f2eabb8cbb44cbebcc0a6ccd8e5836b79ba3f23e37f20054c9e"}
Oct 30 22:22:50.488885 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:22:50.488802    4473 scope.go:122] "RemoveContainer" containerID="153757e7bbcb55858fe522c63c89b4d27454da55a5d79eafc8df23c63b75eff3"
Oct 30 22:22:50.504400 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:22:50.504378    4473 scope.go:122] "RemoveContainer" containerID="153757e7bbcb55858fe522c63c89b4d27454da55a5d79eafc8df23c63b75eff3"
Oct 30 22:22:50.504772 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:22:50.504751    4473 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"153757e7bbcb55858fe522c63c89b4d27454da55a5d79eafc8df23c63b75eff3\": container with ID starting with 153757e7bbcb55858fe522c63c89b4d27454da55a5d79eafc8df23c63b75eff3 not found: ID does not exist" containerID="153757e7bbcb55858fe522c63c89b4d27454da55a5d79eafc8df23c63b75eff3"
Oct 30 22:22:50.504845 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:22:50.504786    4473 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"153757e7bbcb55858fe522c63c89b4d27454da55a5d79eafc8df23c63b75eff3"} err="failed to get container status \"153757e7bbcb55858fe522c63c89b4d27454da55a5d79eafc8df23c63b75eff3\": rpc error: code = NotFound desc = could not find container \"153757e7bbcb55858fe522c63c89b4d27454da55a5d79eafc8df23c63b75eff3\": container with ID starting with 153757e7bbcb55858fe522c63c89b4d27454da55a5d79eafc8df23c63b75eff3 not found: ID does not exist"
Oct 30 22:22:50.510112 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:22:50.510082    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8b90548f-3da8-42df-99d0-d5eb415a4af3-istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/8b90548f-3da8-42df-99d0-d5eb415a4af3-istio-csr-ca-configmap\") pod \"8b90548f-3da8-42df-99d0-d5eb415a4af3\" (UID: \"8b90548f-3da8-42df-99d0-d5eb415a4af3\") "
Oct 30 22:22:50.510227 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:22:50.510162    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8b90548f-3da8-42df-99d0-d5eb415a4af3-kube-api-access-trzvr\" (UniqueName: \"kubernetes.io/projected/8b90548f-3da8-42df-99d0-d5eb415a4af3-kube-api-access-trzvr\") pod \"8b90548f-3da8-42df-99d0-d5eb415a4af3\" (UID: \"8b90548f-3da8-42df-99d0-d5eb415a4af3\") "
Oct 30 22:22:50.510292 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:22:50.510247    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8b90548f-3da8-42df-99d0-d5eb415a4af3-local-certs\" (UniqueName: \"kubernetes.io/empty-dir/8b90548f-3da8-42df-99d0-d5eb415a4af3-local-certs\") pod \"8b90548f-3da8-42df-99d0-d5eb415a4af3\" (UID: \"8b90548f-3da8-42df-99d0-d5eb415a4af3\") "
Oct 30 22:22:50.510356 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:22:50.510296    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8b90548f-3da8-42df-99d0-d5eb415a4af3-istio-token\" (UniqueName: \"kubernetes.io/projected/8b90548f-3da8-42df-99d0-d5eb415a4af3-istio-token\") pod \"8b90548f-3da8-42df-99d0-d5eb415a4af3\" (UID: \"8b90548f-3da8-42df-99d0-d5eb415a4af3\") "
Oct 30 22:22:50.510356 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:22:50.510314    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8b90548f-3da8-42df-99d0-d5eb415a4af3-cacerts\" (UniqueName: \"kubernetes.io/secret/8b90548f-3da8-42df-99d0-d5eb415a4af3-cacerts\") pod \"8b90548f-3da8-42df-99d0-d5eb415a4af3\" (UID: \"8b90548f-3da8-42df-99d0-d5eb415a4af3\") "
Oct 30 22:22:50.510356 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:22:50.510351    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8b90548f-3da8-42df-99d0-d5eb415a4af3-istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8b90548f-3da8-42df-99d0-d5eb415a4af3-istio-kubeconfig\") pod \"8b90548f-3da8-42df-99d0-d5eb415a4af3\" (UID: \"8b90548f-3da8-42df-99d0-d5eb415a4af3\") "
Oct 30 22:22:50.510522 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:22:50.510376    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8b90548f-3da8-42df-99d0-d5eb415a4af3-istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/8b90548f-3da8-42df-99d0-d5eb415a4af3-istio-csr-dns-cert\") pod \"8b90548f-3da8-42df-99d0-d5eb415a4af3\" (UID: \"8b90548f-3da8-42df-99d0-d5eb415a4af3\") "
Oct 30 22:22:50.510522 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:22:50.510474    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8b90548f-3da8-42df-99d0-d5eb415a4af3-istio-csr-ca-configmap" pod "8b90548f-3da8-42df-99d0-d5eb415a4af3" (UID: "8b90548f-3da8-42df-99d0-d5eb415a4af3"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:22:50.510762 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:22:50.510733    4473 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/8b90548f-3da8-42df-99d0-d5eb415a4af3-istio-csr-ca-configmap\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:22:50.512648 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:22:50.512615    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8b90548f-3da8-42df-99d0-d5eb415a4af3-cacerts" pod "8b90548f-3da8-42df-99d0-d5eb415a4af3" (UID: "8b90548f-3da8-42df-99d0-d5eb415a4af3"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:22:50.512805 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:22:50.512779    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8b90548f-3da8-42df-99d0-d5eb415a4af3-istio-kubeconfig" pod "8b90548f-3da8-42df-99d0-d5eb415a4af3" (UID: "8b90548f-3da8-42df-99d0-d5eb415a4af3"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:22:50.513146 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:22:50.513115    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8b90548f-3da8-42df-99d0-d5eb415a4af3-kube-api-access-trzvr" pod "8b90548f-3da8-42df-99d0-d5eb415a4af3" (UID: "8b90548f-3da8-42df-99d0-d5eb415a4af3"). InnerVolumeSpecName "kube-api-access-trzvr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:22:50.513146 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:22:50.513134    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8b90548f-3da8-42df-99d0-d5eb415a4af3-istio-token" pod "8b90548f-3da8-42df-99d0-d5eb415a4af3" (UID: "8b90548f-3da8-42df-99d0-d5eb415a4af3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:22:50.513299 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:22:50.513158    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b90548f-3da8-42df-99d0-d5eb415a4af3-local-certs" pod "8b90548f-3da8-42df-99d0-d5eb415a4af3" (UID: "8b90548f-3da8-42df-99d0-d5eb415a4af3"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:22:50.513299 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:22:50.513237    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8b90548f-3da8-42df-99d0-d5eb415a4af3-istio-csr-dns-cert" pod "8b90548f-3da8-42df-99d0-d5eb415a4af3" (UID: "8b90548f-3da8-42df-99d0-d5eb415a4af3"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:22:50.612229 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:22:50.612197    4473 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/8b90548f-3da8-42df-99d0-d5eb415a4af3-local-certs\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:22:50.612229 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:22:50.612226    4473 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8b90548f-3da8-42df-99d0-d5eb415a4af3-istio-token\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:22:50.612229 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:22:50.612234    4473 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/8b90548f-3da8-42df-99d0-d5eb415a4af3-cacerts\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:22:50.612474 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:22:50.612242    4473 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8b90548f-3da8-42df-99d0-d5eb415a4af3-istio-kubeconfig\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:22:50.612474 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:22:50.612251    4473 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/8b90548f-3da8-42df-99d0-d5eb415a4af3-istio-csr-dns-cert\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:22:50.612474 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:22:50.612260    4473 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-trzvr\" (UniqueName: \"kubernetes.io/projected/8b90548f-3da8-42df-99d0-d5eb415a4af3-kube-api-access-trzvr\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:22:50.910910 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:22:50.910873    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-lxkmp"]
Oct 30 22:22:50.961363 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:22:50.961326    4473 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-lxkmp"]
Oct 30 22:22:52.356013 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:22:52.355976    4473 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8b90548f-3da8-42df-99d0-d5eb415a4af3" path="/var/lib/kubelet/pods/8b90548f-3da8-42df-99d0-d5eb415a4af3/volumes"
Oct 30 22:23:20.324644 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:23:20.324605    4473 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fncvg" containerName="extractor"
Oct 30 22:23:39.051376 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:23:39.051335    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-9383b377-tlnrt"]
Oct 30 22:23:39.054876 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:23:39.054852    4473 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-9383b377-tlnrt"]
Oct 30 22:23:40.355764 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:23:40.355734    4473 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0b139aed-161c-425c-a63c-59441bcdaa54" path="/var/lib/kubelet/pods/0b139aed-161c-425c-a63c-59441bcdaa54/volumes"
Oct 30 22:23:50.322424 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:23:50.322340    4473 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fncvg" containerName="extractor"
Oct 30 22:24:20.324286 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:24:20.324248    4473 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fncvg" containerName="extractor"
Oct 30 22:24:37.495392 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:24:37.495355    4473 scope.go:122] "RemoveContainer" containerID="249747718be503b6a204625d92a84b8de2c4e38cff4b25d6ffc3df8de037595f"
Oct 30 22:24:50.427678 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:24:50.427599    4473 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fncvg" containerName="extractor"
Oct 30 22:25:02.047403 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:25:02.047366    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-6826e712-c97kp"]
Oct 30 22:25:02.097490 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:25:02.097457    4473 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-6826e712-c97kp"]
Oct 30 22:25:02.355813 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:25:02.355735    4473 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0df24205-ae49-4abd-abf8-afa8955f7d61" path="/var/lib/kubelet/pods/0df24205-ae49-4abd-abf8-afa8955f7d61/volumes"
Oct 30 22:25:20.325637 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:25:20.325557    4473 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fncvg" containerName="extractor"
Oct 30 22:25:50.323408 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:25:50.323372    4473 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fncvg" containerName="extractor"
Oct 30 22:26:19.049035 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:26:19.048993    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-459d3dc2-r7c8g"]
Oct 30 22:26:19.098661 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:26:19.098625    4473 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-459d3dc2-r7c8g"]
Oct 30 22:26:20.324368 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:26:20.324332    4473 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fncvg" containerName="extractor"
Oct 30 22:26:20.355654 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:26:20.355616    4473 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3e9f27bc-2bf5-4b5d-b6b1-d63c5271624e" path="/var/lib/kubelet/pods/3e9f27bc-2bf5-4b5d-b6b1-d63c5271624e/volumes"
Oct 30 22:26:37.522800 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:26:37.522767    4473 scope.go:122] "RemoveContainer" containerID="17a968ba4f8f54f0d8920c412953a83651300b237a11942abcb557c7b47e29dc"
Oct 30 22:26:50.323258 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:26:50.323168    4473 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fncvg" containerName="extractor"
Oct 30 22:27:08.579444 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:27:08.579366    4473 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-5b9r4 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.146:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 22:27:08.580315 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:27:08.580267    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-5b9r4" podUID="1557635a-ac42-4385-99f3-879641eb0ccc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.146:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:27:20.324248 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:27:20.324208    4473 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fncvg" containerName="extractor"
Oct 30 22:27:37.548541 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:27:37.548507    4473 scope.go:122] "RemoveContainer" containerID="2ca89a3ebd72788149df78b1b7805151a280ac71c2f4451d81336ff7354ba9a1"
Oct 30 22:27:50.325005 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:27:50.324972    4473 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fncvg" containerName="extractor"
Oct 30 22:28:20.323131 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:28:20.323042    4473 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fncvg" containerName="extractor"
Oct 30 22:28:34.748648 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:28:34.748607    4473 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-single-main-9513bb-openshift-default-67677488d5j5bcz"]
Oct 30 22:28:34.749163 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:28:34.749105    4473 cpu_manager.go:414] "containerMap: removing container" podUID="8b90548f-3da8-42df-99d0-d5eb415a4af3" containerName="discovery"
Oct 30 22:28:34.749163 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:28:34.749119    4473 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8b90548f-3da8-42df-99d0-d5eb415a4af3" containerName="discovery"
Oct 30 22:28:34.749245 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:28:34.749198    4473 memory_manager.go:365] "RemoveStaleState removing state" podUID="8b90548f-3da8-42df-99d0-d5eb415a4af3" containerName="discovery"
Oct 30 22:28:34.752362 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:28:34.752342    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-single-main-9513bb-openshift-default-67677488d5j5bcz"
Oct 30 22:28:34.810047 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:28:34.810017    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-single-main-9513bb-openshift-default-dockercfg-c2rj4\""
Oct 30 22:28:34.815261 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:28:34.815168    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-single-main-9513bb-openshift-default-67677488d5j5bcz"]
Oct 30 22:28:34.822597 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:28:34.822551    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a63d380-983c-49ae-a8a0-c8e47ead8d1a-credential-socket\") pod \"e2e-worker-single-main-9513bb-openshift-default-67677488d5j5bcz\" (UID: \"3a63d380-983c-49ae-a8a0-c8e47ead8d1a\") " pod="openshift-ingress/e2e-worker-single-main-9513bb-openshift-default-67677488d5j5bcz"
Oct 30 22:28:34.822779 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:28:34.822618    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3a63d380-983c-49ae-a8a0-c8e47ead8d1a-istio-podinfo\") pod \"e2e-worker-single-main-9513bb-openshift-default-67677488d5j5bcz\" (UID: \"3a63d380-983c-49ae-a8a0-c8e47ead8d1a\") " pod="openshift-ingress/e2e-worker-single-main-9513bb-openshift-default-67677488d5j5bcz"
Oct 30 22:28:34.822779 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:28:34.822657    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w2m7r\" (UniqueName: \"kubernetes.io/projected/3a63d380-983c-49ae-a8a0-c8e47ead8d1a-kube-api-access-w2m7r\") pod \"e2e-worker-single-main-9513bb-openshift-default-67677488d5j5bcz\" (UID: \"3a63d380-983c-49ae-a8a0-c8e47ead8d1a\") " pod="openshift-ingress/e2e-worker-single-main-9513bb-openshift-default-67677488d5j5bcz"
Oct 30 22:28:34.822779 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:28:34.822699    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3a63d380-983c-49ae-a8a0-c8e47ead8d1a-istiod-ca-cert\") pod \"e2e-worker-single-main-9513bb-openshift-default-67677488d5j5bcz\" (UID: \"3a63d380-983c-49ae-a8a0-c8e47ead8d1a\") " pod="openshift-ingress/e2e-worker-single-main-9513bb-openshift-default-67677488d5j5bcz"
Oct 30 22:28:34.822779 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:28:34.822743    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3a63d380-983c-49ae-a8a0-c8e47ead8d1a-istio-envoy\") pod \"e2e-worker-single-main-9513bb-openshift-default-67677488d5j5bcz\" (UID: \"3a63d380-983c-49ae-a8a0-c8e47ead8d1a\") " pod="openshift-ingress/e2e-worker-single-main-9513bb-openshift-default-67677488d5j5bcz"
Oct 30 22:28:34.823044 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:28:34.822799    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a63d380-983c-49ae-a8a0-c8e47ead8d1a-workload-socket\") pod \"e2e-worker-single-main-9513bb-openshift-default-67677488d5j5bcz\" (UID: \"3a63d380-983c-49ae-a8a0-c8e47ead8d1a\") " pod="openshift-ingress/e2e-worker-single-main-9513bb-openshift-default-67677488d5j5bcz"
Oct 30 22:28:34.823044 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:28:34.822844    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3a63d380-983c-49ae-a8a0-c8e47ead8d1a-workload-certs\") pod \"e2e-worker-single-main-9513bb-openshift-default-67677488d5j5bcz\" (UID: \"3a63d380-983c-49ae-a8a0-c8e47ead8d1a\") " pod="openshift-ingress/e2e-worker-single-main-9513bb-openshift-default-67677488d5j5bcz"
Oct 30 22:28:34.823044 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:28:34.822870    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3a63d380-983c-49ae-a8a0-c8e47ead8d1a-istio-data\") pod \"e2e-worker-single-main-9513bb-openshift-default-67677488d5j5bcz\" (UID: \"3a63d380-983c-49ae-a8a0-c8e47ead8d1a\") " pod="openshift-ingress/e2e-worker-single-main-9513bb-openshift-default-67677488d5j5bcz"
Oct 30 22:28:34.823044 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:28:34.822976    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3a63d380-983c-49ae-a8a0-c8e47ead8d1a-istio-token\") pod \"e2e-worker-single-main-9513bb-openshift-default-67677488d5j5bcz\" (UID: \"3a63d380-983c-49ae-a8a0-c8e47ead8d1a\") " pod="openshift-ingress/e2e-worker-single-main-9513bb-openshift-default-67677488d5j5bcz"
Oct 30 22:28:34.923810 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:28:34.923772    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a63d380-983c-49ae-a8a0-c8e47ead8d1a-credential-socket\") pod \"e2e-worker-single-main-9513bb-openshift-default-67677488d5j5bcz\" (UID: \"3a63d380-983c-49ae-a8a0-c8e47ead8d1a\") " pod="openshift-ingress/e2e-worker-single-main-9513bb-openshift-default-67677488d5j5bcz"
Oct 30 22:28:34.923810 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:28:34.923808    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3a63d380-983c-49ae-a8a0-c8e47ead8d1a-istio-podinfo\") pod \"e2e-worker-single-main-9513bb-openshift-default-67677488d5j5bcz\" (UID: \"3a63d380-983c-49ae-a8a0-c8e47ead8d1a\") " pod="openshift-ingress/e2e-worker-single-main-9513bb-openshift-default-67677488d5j5bcz"
Oct 30 22:28:34.924118 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:28:34.923830    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w2m7r\" (UniqueName: \"kubernetes.io/projected/3a63d380-983c-49ae-a8a0-c8e47ead8d1a-kube-api-access-w2m7r\") pod \"e2e-worker-single-main-9513bb-openshift-default-67677488d5j5bcz\" (UID: \"3a63d380-983c-49ae-a8a0-c8e47ead8d1a\") " pod="openshift-ingress/e2e-worker-single-main-9513bb-openshift-default-67677488d5j5bcz"
Oct 30 22:28:34.924118 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:28:34.923854    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3a63d380-983c-49ae-a8a0-c8e47ead8d1a-istiod-ca-cert\") pod \"e2e-worker-single-main-9513bb-openshift-default-67677488d5j5bcz\" (UID: \"3a63d380-983c-49ae-a8a0-c8e47ead8d1a\") " pod="openshift-ingress/e2e-worker-single-main-9513bb-openshift-default-67677488d5j5bcz"
Oct 30 22:28:34.924118 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:28:34.923889    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3a63d380-983c-49ae-a8a0-c8e47ead8d1a-istio-envoy\") pod \"e2e-worker-single-main-9513bb-openshift-default-67677488d5j5bcz\" (UID: \"3a63d380-983c-49ae-a8a0-c8e47ead8d1a\") " pod="openshift-ingress/e2e-worker-single-main-9513bb-openshift-default-67677488d5j5bcz"
Oct 30 22:28:34.924118 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:28:34.923951    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a63d380-983c-49ae-a8a0-c8e47ead8d1a-workload-socket\") pod \"e2e-worker-single-main-9513bb-openshift-default-67677488d5j5bcz\" (UID: \"3a63d380-983c-49ae-a8a0-c8e47ead8d1a\") " pod="openshift-ingress/e2e-worker-single-main-9513bb-openshift-default-67677488d5j5bcz"
Oct 30 22:28:34.924118 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:28:34.923995    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3a63d380-983c-49ae-a8a0-c8e47ead8d1a-workload-certs\") pod \"e2e-worker-single-main-9513bb-openshift-default-67677488d5j5bcz\" (UID: \"3a63d380-983c-49ae-a8a0-c8e47ead8d1a\") " pod="openshift-ingress/e2e-worker-single-main-9513bb-openshift-default-67677488d5j5bcz"
Oct 30 22:28:34.924118 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:28:34.924019    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3a63d380-983c-49ae-a8a0-c8e47ead8d1a-istio-data\") pod \"e2e-worker-single-main-9513bb-openshift-default-67677488d5j5bcz\" (UID: \"3a63d380-983c-49ae-a8a0-c8e47ead8d1a\") " pod="openshift-ingress/e2e-worker-single-main-9513bb-openshift-default-67677488d5j5bcz"
Oct 30 22:28:34.924118 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:28:34.924079    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3a63d380-983c-49ae-a8a0-c8e47ead8d1a-istio-token\") pod \"e2e-worker-single-main-9513bb-openshift-default-67677488d5j5bcz\" (UID: \"3a63d380-983c-49ae-a8a0-c8e47ead8d1a\") " pod="openshift-ingress/e2e-worker-single-main-9513bb-openshift-default-67677488d5j5bcz"
Oct 30 22:28:34.924506 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:28:34.924339    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a63d380-983c-49ae-a8a0-c8e47ead8d1a-credential-socket\") pod \"e2e-worker-single-main-9513bb-openshift-default-67677488d5j5bcz\" (UID: \"3a63d380-983c-49ae-a8a0-c8e47ead8d1a\") " pod="openshift-ingress/e2e-worker-single-main-9513bb-openshift-default-67677488d5j5bcz"
Oct 30 22:28:34.924506 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:28:34.924416    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a63d380-983c-49ae-a8a0-c8e47ead8d1a-workload-socket\") pod \"e2e-worker-single-main-9513bb-openshift-default-67677488d5j5bcz\" (UID: \"3a63d380-983c-49ae-a8a0-c8e47ead8d1a\") " pod="openshift-ingress/e2e-worker-single-main-9513bb-openshift-default-67677488d5j5bcz"
Oct 30 22:28:34.924617 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:28:34.924525    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3a63d380-983c-49ae-a8a0-c8e47ead8d1a-istio-data\") pod \"e2e-worker-single-main-9513bb-openshift-default-67677488d5j5bcz\" (UID: \"3a63d380-983c-49ae-a8a0-c8e47ead8d1a\") " pod="openshift-ingress/e2e-worker-single-main-9513bb-openshift-default-67677488d5j5bcz"
Oct 30 22:28:34.924617 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:28:34.924597    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3a63d380-983c-49ae-a8a0-c8e47ead8d1a-workload-certs\") pod \"e2e-worker-single-main-9513bb-openshift-default-67677488d5j5bcz\" (UID: \"3a63d380-983c-49ae-a8a0-c8e47ead8d1a\") " pod="openshift-ingress/e2e-worker-single-main-9513bb-openshift-default-67677488d5j5bcz"
Oct 30 22:28:34.924899 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:28:34.924706    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3a63d380-983c-49ae-a8a0-c8e47ead8d1a-istiod-ca-cert\") pod \"e2e-worker-single-main-9513bb-openshift-default-67677488d5j5bcz\" (UID: \"3a63d380-983c-49ae-a8a0-c8e47ead8d1a\") " pod="openshift-ingress/e2e-worker-single-main-9513bb-openshift-default-67677488d5j5bcz"
Oct 30 22:28:34.926444 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:28:34.926417    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3a63d380-983c-49ae-a8a0-c8e47ead8d1a-istio-envoy\") pod \"e2e-worker-single-main-9513bb-openshift-default-67677488d5j5bcz\" (UID: \"3a63d380-983c-49ae-a8a0-c8e47ead8d1a\") " pod="openshift-ingress/e2e-worker-single-main-9513bb-openshift-default-67677488d5j5bcz"
Oct 30 22:28:34.926704 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:28:34.926683    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3a63d380-983c-49ae-a8a0-c8e47ead8d1a-istio-podinfo\") pod \"e2e-worker-single-main-9513bb-openshift-default-67677488d5j5bcz\" (UID: \"3a63d380-983c-49ae-a8a0-c8e47ead8d1a\") " pod="openshift-ingress/e2e-worker-single-main-9513bb-openshift-default-67677488d5j5bcz"
Oct 30 22:28:34.981913 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:28:34.981879    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3a63d380-983c-49ae-a8a0-c8e47ead8d1a-istio-token\") pod \"e2e-worker-single-main-9513bb-openshift-default-67677488d5j5bcz\" (UID: \"3a63d380-983c-49ae-a8a0-c8e47ead8d1a\") " pod="openshift-ingress/e2e-worker-single-main-9513bb-openshift-default-67677488d5j5bcz"
Oct 30 22:28:34.982089 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:28:34.982015    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w2m7r\" (UniqueName: \"kubernetes.io/projected/3a63d380-983c-49ae-a8a0-c8e47ead8d1a-kube-api-access-w2m7r\") pod \"e2e-worker-single-main-9513bb-openshift-default-67677488d5j5bcz\" (UID: \"3a63d380-983c-49ae-a8a0-c8e47ead8d1a\") " pod="openshift-ingress/e2e-worker-single-main-9513bb-openshift-default-67677488d5j5bcz"
Oct 30 22:28:35.064957 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:28:35.064844    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-single-main-9513bb-openshift-default-67677488d5j5bcz"
Oct 30 22:28:35.489700 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:28:35.489674    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-single-main-9513bb-openshift-default-67677488d5j5bcz"]
Oct 30 22:28:35.491412 ip-10-0-140-18 kubenswrapper[4473]: W1030 22:28:35.491383    4473 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a63d380_983c_49ae_a8a0_c8e47ead8d1a.slice/crio-1393701dcdb78fd5a8785d9a1f8e97c37aa7cc2c01323cea529751493dae53e8 WatchSource:0}: Error finding container 1393701dcdb78fd5a8785d9a1f8e97c37aa7cc2c01323cea529751493dae53e8: Status 404 returned error can't find the container with id 1393701dcdb78fd5a8785d9a1f8e97c37aa7cc2c01323cea529751493dae53e8
Oct 30 22:28:35.493494 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:28:35.493461    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:28:35.493561 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:28:35.493527    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:28:35.493561 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:28:35.493556    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:28:36.459630 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:28:36.459590    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-9513bb-openshift-default-67677488d5j5bcz" event={"ID":"3a63d380-983c-49ae-a8a0-c8e47ead8d1a","Type":"ContainerStarted","Data":"43b8e3bc84ec1dad5a9911989bde0ec6dfde51a28cd71a635605c074420282de"}
Oct 30 22:28:36.460086 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:28:36.459641    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-9513bb-openshift-default-67677488d5j5bcz" event={"ID":"3a63d380-983c-49ae-a8a0-c8e47ead8d1a","Type":"ContainerStarted","Data":"1393701dcdb78fd5a8785d9a1f8e97c37aa7cc2c01323cea529751493dae53e8"}
Oct 30 22:28:36.632267 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:28:36.632217    4473 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-single-main-9513bb-openshift-default-67677488d5j5bcz" podStartSLOduration=2.632204198 podStartE2EDuration="2.632204198s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:28:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:28:36.631289699 +0000 UTC m=+2762.404658187" watchObservedRunningTime="2026-10-30 22:28:36.632204198 +0000 UTC m=+2762.405572759"
Oct 30 22:28:37.065675 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:28:37.065643    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-single-main-9513bb-openshift-default-67677488d5j5bcz"
Oct 30 22:28:37.070872 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:28:37.070838    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-single-main-9513bb-openshift-default-67677488d5j5bcz"
Oct 30 22:28:37.464804 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:28:37.464770    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-single-main-9513bb-openshift-default-67677488d5j5bcz"
Oct 30 22:28:37.465789 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:28:37.465770    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-single-main-9513bb-openshift-default-67677488d5j5bcz"
Oct 30 22:28:43.404130 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:28:43.404098    4473 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-9513bb-97d84485fdzpmr"]
Oct 30 22:28:43.408092 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:28:43.408061    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-single-main-9513bb-97d84485fdzpmr"
Oct 30 22:28:43.460905 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:28:43.460868    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-9513bb-97d84485fdzpmr"]
Oct 30 22:28:43.509464 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:28:43.509425    4473 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-single-main-9513bb\""
Oct 30 22:28:43.509638 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:28:43.509477    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-single-m-52792b8f-dockercfg-ltcv4\""
Oct 30 22:28:43.516832 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:28:43.516804    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p4t8j\" (UniqueName: \"kubernetes.io/projected/525bfdc5-b511-42c1-890e-ead9010f1b4b-kube-api-access-p4t8j\") pod \"payload-processing-e2e-worker-single-main-9513bb-97d84485fdzpmr\" (UID: \"525bfdc5-b511-42c1-890e-ead9010f1b4b\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-9513bb-97d84485fdzpmr"
Oct 30 22:28:43.516991 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:28:43.516971    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/525bfdc5-b511-42c1-890e-ead9010f1b4b-plugins-config-volume\") pod \"payload-processing-e2e-worker-single-main-9513bb-97d84485fdzpmr\" (UID: \"525bfdc5-b511-42c1-890e-ead9010f1b4b\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-9513bb-97d84485fdzpmr"
Oct 30 22:28:43.565883 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:28:43.565844    4473 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-9513bb-5dcb9zwg2c"]
Oct 30 22:28:43.569490 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:28:43.569474    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-9513bb-5dcb9zwg2c"
Oct 30 22:28:43.618561 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:28:43.618525    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zdjf9\" (UniqueName: \"kubernetes.io/projected/4d6c8b70-a3dc-42fc-bbfb-3937c3f349cb-kube-api-access-zdjf9\") pod \"payload-pre-processing-e2e-worker-single-main-9513bb-5dcb9zwg2c\" (UID: \"4d6c8b70-a3dc-42fc-bbfb-3937c3f349cb\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-9513bb-5dcb9zwg2c"
Oct 30 22:28:43.618730 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:28:43.618583    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4d6c8b70-a3dc-42fc-bbfb-3937c3f349cb-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-single-main-9513bb-5dcb9zwg2c\" (UID: \"4d6c8b70-a3dc-42fc-bbfb-3937c3f349cb\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-9513bb-5dcb9zwg2c"
Oct 30 22:28:43.618730 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:28:43.618682    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/525bfdc5-b511-42c1-890e-ead9010f1b4b-plugins-config-volume\") pod \"payload-processing-e2e-worker-single-main-9513bb-97d84485fdzpmr\" (UID: \"525bfdc5-b511-42c1-890e-ead9010f1b4b\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-9513bb-97d84485fdzpmr"
Oct 30 22:28:43.618805 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:28:43.618736    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p4t8j\" (UniqueName: \"kubernetes.io/projected/525bfdc5-b511-42c1-890e-ead9010f1b4b-kube-api-access-p4t8j\") pod \"payload-processing-e2e-worker-single-main-9513bb-97d84485fdzpmr\" (UID: \"525bfdc5-b511-42c1-890e-ead9010f1b4b\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-9513bb-97d84485fdzpmr"
Oct 30 22:28:43.619413 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:28:43.619390    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/525bfdc5-b511-42c1-890e-ead9010f1b4b-plugins-config-volume\") pod \"payload-processing-e2e-worker-single-main-9513bb-97d84485fdzpmr\" (UID: \"525bfdc5-b511-42c1-890e-ead9010f1b4b\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-9513bb-97d84485fdzpmr"
Oct 30 22:28:43.627559 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:28:43.627534    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-9513bb-5dcb9zwg2c"]
Oct 30 22:28:43.677159 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:28:43.677092    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p4t8j\" (UniqueName: \"kubernetes.io/projected/525bfdc5-b511-42c1-890e-ead9010f1b4b-kube-api-access-p4t8j\") pod \"payload-processing-e2e-worker-single-main-9513bb-97d84485fdzpmr\" (UID: \"525bfdc5-b511-42c1-890e-ead9010f1b4b\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-9513bb-97d84485fdzpmr"
Oct 30 22:28:43.720014 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:28:43.719981    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zdjf9\" (UniqueName: \"kubernetes.io/projected/4d6c8b70-a3dc-42fc-bbfb-3937c3f349cb-kube-api-access-zdjf9\") pod \"payload-pre-processing-e2e-worker-single-main-9513bb-5dcb9zwg2c\" (UID: \"4d6c8b70-a3dc-42fc-bbfb-3937c3f349cb\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-9513bb-5dcb9zwg2c"
Oct 30 22:28:43.720189 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:28:43.720042    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4d6c8b70-a3dc-42fc-bbfb-3937c3f349cb-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-single-main-9513bb-5dcb9zwg2c\" (UID: \"4d6c8b70-a3dc-42fc-bbfb-3937c3f349cb\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-9513bb-5dcb9zwg2c"
Oct 30 22:28:43.720781 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:28:43.720759    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4d6c8b70-a3dc-42fc-bbfb-3937c3f349cb-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-single-main-9513bb-5dcb9zwg2c\" (UID: \"4d6c8b70-a3dc-42fc-bbfb-3937c3f349cb\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-9513bb-5dcb9zwg2c"
Oct 30 22:28:43.722821 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:28:43.722796    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-single-main-9513bb-97d84485fdzpmr"
Oct 30 22:28:43.779612 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:28:43.779548    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zdjf9\" (UniqueName: \"kubernetes.io/projected/4d6c8b70-a3dc-42fc-bbfb-3937c3f349cb-kube-api-access-zdjf9\") pod \"payload-pre-processing-e2e-worker-single-main-9513bb-5dcb9zwg2c\" (UID: \"4d6c8b70-a3dc-42fc-bbfb-3937c3f349cb\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-9513bb-5dcb9zwg2c"
Oct 30 22:28:43.883095 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:28:43.883057    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-9513bb-5dcb9zwg2c"
Oct 30 22:28:44.093433 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:28:44.093403    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-9513bb-97d84485fdzpmr"]
Oct 30 22:28:44.094333 ip-10-0-140-18 kubenswrapper[4473]: W1030 22:28:44.094302    4473 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod525bfdc5_b511_42c1_890e_ead9010f1b4b.slice/crio-d19b1a8103479da5d047c72e46a08dddd9e77f9f7551d6aaa25b697f150d7cac WatchSource:0}: Error finding container d19b1a8103479da5d047c72e46a08dddd9e77f9f7551d6aaa25b697f150d7cac: Status 404 returned error can't find the container with id d19b1a8103479da5d047c72e46a08dddd9e77f9f7551d6aaa25b697f150d7cac
Oct 30 22:28:44.256685 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:28:44.256659    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-9513bb-5dcb9zwg2c"]
Oct 30 22:28:44.257863 ip-10-0-140-18 kubenswrapper[4473]: W1030 22:28:44.257836    4473 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d6c8b70_a3dc_42fc_bbfb_3937c3f349cb.slice/crio-b9768d35454a64d9df4461e7dab187ab96ff42f9027bdc7cf1ab980d793f1e37 WatchSource:0}: Error finding container b9768d35454a64d9df4461e7dab187ab96ff42f9027bdc7cf1ab980d793f1e37: Status 404 returned error can't find the container with id b9768d35454a64d9df4461e7dab187ab96ff42f9027bdc7cf1ab980d793f1e37
Oct 30 22:28:44.501354 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:28:44.501314    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-single-main-9513bb-97d84485fdzpmr" event={"ID":"525bfdc5-b511-42c1-890e-ead9010f1b4b","Type":"ContainerStarted","Data":"4de858a2777cf28178f4b1041949de1e2c4c90158f782b92bb5c66068f43374b"}
Oct 30 22:28:44.501830 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:28:44.501373    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-single-main-9513bb-97d84485fdzpmr" event={"ID":"525bfdc5-b511-42c1-890e-ead9010f1b4b","Type":"ContainerStarted","Data":"d19b1a8103479da5d047c72e46a08dddd9e77f9f7551d6aaa25b697f150d7cac"}
Oct 30 22:28:44.501830 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:28:44.501400    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-single-main-9513bb-97d84485fdzpmr"
Oct 30 22:28:44.502964 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:28:44.502910    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-9513bb-5dcb9zwg2c" event={"ID":"4d6c8b70-a3dc-42fc-bbfb-3937c3f349cb","Type":"ContainerStarted","Data":"5c6916f1869ac777b433fddc8f1c4826eb28e25d4d429b3dfab5d9e1aa1047ca"}
Oct 30 22:28:44.503067 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:28:44.502983    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-9513bb-5dcb9zwg2c" event={"ID":"4d6c8b70-a3dc-42fc-bbfb-3937c3f349cb","Type":"ContainerStarted","Data":"b9768d35454a64d9df4461e7dab187ab96ff42f9027bdc7cf1ab980d793f1e37"}
Oct 30 22:28:44.503067 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:28:44.503004    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-9513bb-5dcb9zwg2c"
Oct 30 22:28:44.620634 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:28:44.620578    4473 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-single-main-9513bb-97d84485fdzpmr" podStartSLOduration=1.620563687 podStartE2EDuration="1.620563687s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:28:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:28:44.620501374 +0000 UTC m=+2770.393869861" watchObservedRunningTime="2026-10-30 22:28:44.620563687 +0000 UTC m=+2770.393932174"
Oct 30 22:28:44.737293 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:28:44.737186    4473 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-9513bb-5dcb9zwg2c" podStartSLOduration=1.737165828 podStartE2EDuration="1.737165828s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:28:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:28:44.736487152 +0000 UTC m=+2770.509855643" watchObservedRunningTime="2026-10-30 22:28:44.737165828 +0000 UTC m=+2770.510534316"
Oct 30 22:28:45.999697 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:28:45.999660    4473 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-9513bb-58f7d87d8d-7bcdq"]
Oct 30 22:28:46.003590 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:28:46.003569    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-9513bb-58f7d87d8d-7bcdq"
Oct 30 22:28:46.056946 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:28:46.056890    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-single-main-9513bb-serving-cert\""
Oct 30 22:28:46.109853 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:28:46.109815    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-9513bb-58f7d87d8d-7bcdq"]
Oct 30 22:28:46.145310 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:28:46.145275    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pt5w6\" (UniqueName: \"kubernetes.io/projected/db69aa0b-8205-4c19-a6c3-838ae5580d0d-kube-api-access-pt5w6\") pod \"maas-api-e2e-worker-single-main-9513bb-58f7d87d8d-7bcdq\" (UID: \"db69aa0b-8205-4c19-a6c3-838ae5580d0d\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-9513bb-58f7d87d8d-7bcdq"
Oct 30 22:28:46.145507 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:28:46.145416    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/db69aa0b-8205-4c19-a6c3-838ae5580d0d-metrics-certs\") pod \"maas-api-e2e-worker-single-main-9513bb-58f7d87d8d-7bcdq\" (UID: \"db69aa0b-8205-4c19-a6c3-838ae5580d0d\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-9513bb-58f7d87d8d-7bcdq"
Oct 30 22:28:46.145507 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:28:46.145469    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/db69aa0b-8205-4c19-a6c3-838ae5580d0d-openshift-service-ca\") pod \"maas-api-e2e-worker-single-main-9513bb-58f7d87d8d-7bcdq\" (UID: \"db69aa0b-8205-4c19-a6c3-838ae5580d0d\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-9513bb-58f7d87d8d-7bcdq"
Oct 30 22:28:46.145507 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:28:46.145505    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/db69aa0b-8205-4c19-a6c3-838ae5580d0d-maas-api-tls\") pod \"maas-api-e2e-worker-single-main-9513bb-58f7d87d8d-7bcdq\" (UID: \"db69aa0b-8205-4c19-a6c3-838ae5580d0d\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-9513bb-58f7d87d8d-7bcdq"
Oct 30 22:28:46.247016 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:28:46.246973    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/db69aa0b-8205-4c19-a6c3-838ae5580d0d-metrics-certs\") pod \"maas-api-e2e-worker-single-main-9513bb-58f7d87d8d-7bcdq\" (UID: \"db69aa0b-8205-4c19-a6c3-838ae5580d0d\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-9513bb-58f7d87d8d-7bcdq"
Oct 30 22:28:46.247194 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:28:46.247028    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/db69aa0b-8205-4c19-a6c3-838ae5580d0d-openshift-service-ca\") pod \"maas-api-e2e-worker-single-main-9513bb-58f7d87d8d-7bcdq\" (UID: \"db69aa0b-8205-4c19-a6c3-838ae5580d0d\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-9513bb-58f7d87d8d-7bcdq"
Oct 30 22:28:46.247194 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:28:46.247062    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/db69aa0b-8205-4c19-a6c3-838ae5580d0d-maas-api-tls\") pod \"maas-api-e2e-worker-single-main-9513bb-58f7d87d8d-7bcdq\" (UID: \"db69aa0b-8205-4c19-a6c3-838ae5580d0d\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-9513bb-58f7d87d8d-7bcdq"
Oct 30 22:28:46.247194 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:28:46.247111    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pt5w6\" (UniqueName: \"kubernetes.io/projected/db69aa0b-8205-4c19-a6c3-838ae5580d0d-kube-api-access-pt5w6\") pod \"maas-api-e2e-worker-single-main-9513bb-58f7d87d8d-7bcdq\" (UID: \"db69aa0b-8205-4c19-a6c3-838ae5580d0d\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-9513bb-58f7d87d8d-7bcdq"
Oct 30 22:28:46.247821 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:28:46.247796    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/db69aa0b-8205-4c19-a6c3-838ae5580d0d-openshift-service-ca\") pod \"maas-api-e2e-worker-single-main-9513bb-58f7d87d8d-7bcdq\" (UID: \"db69aa0b-8205-4c19-a6c3-838ae5580d0d\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-9513bb-58f7d87d8d-7bcdq"
Oct 30 22:28:46.249948 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:28:46.249866    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/db69aa0b-8205-4c19-a6c3-838ae5580d0d-metrics-certs\") pod \"maas-api-e2e-worker-single-main-9513bb-58f7d87d8d-7bcdq\" (UID: \"db69aa0b-8205-4c19-a6c3-838ae5580d0d\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-9513bb-58f7d87d8d-7bcdq"
Oct 30 22:28:46.249948 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:28:46.249864    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/db69aa0b-8205-4c19-a6c3-838ae5580d0d-maas-api-tls\") pod \"maas-api-e2e-worker-single-main-9513bb-58f7d87d8d-7bcdq\" (UID: \"db69aa0b-8205-4c19-a6c3-838ae5580d0d\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-9513bb-58f7d87d8d-7bcdq"
Oct 30 22:28:46.308016 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:28:46.307987    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pt5w6\" (UniqueName: \"kubernetes.io/projected/db69aa0b-8205-4c19-a6c3-838ae5580d0d-kube-api-access-pt5w6\") pod \"maas-api-e2e-worker-single-main-9513bb-58f7d87d8d-7bcdq\" (UID: \"db69aa0b-8205-4c19-a6c3-838ae5580d0d\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-9513bb-58f7d87d8d-7bcdq"
Oct 30 22:28:46.316764 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:28:46.316740    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-9513bb-58f7d87d8d-7bcdq"
Oct 30 22:28:46.689248 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:28:46.689222    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-9513bb-58f7d87d8d-7bcdq"]
Oct 30 22:28:46.690701 ip-10-0-140-18 kubenswrapper[4473]: W1030 22:28:46.690668    4473 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb69aa0b_8205_4c19_a6c3_838ae5580d0d.slice/crio-4dffd20a5e08979da97c07f4f8384d5ee33b72efa4e697c35ce9a0df617dbbc8 WatchSource:0}: Error finding container 4dffd20a5e08979da97c07f4f8384d5ee33b72efa4e697c35ce9a0df617dbbc8: Status 404 returned error can't find the container with id 4dffd20a5e08979da97c07f4f8384d5ee33b72efa4e697c35ce9a0df617dbbc8
Oct 30 22:28:46.692206 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:28:46.692189    4473 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 22:28:47.518387 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:28:47.518355    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-9513bb-58f7d87d8d-7bcdq" event={"ID":"db69aa0b-8205-4c19-a6c3-838ae5580d0d","Type":"ContainerStarted","Data":"4dffd20a5e08979da97c07f4f8384d5ee33b72efa4e697c35ce9a0df617dbbc8"}
Oct 30 22:28:48.524384 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:28:48.524346    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-9513bb-58f7d87d8d-7bcdq" event={"ID":"db69aa0b-8205-4c19-a6c3-838ae5580d0d","Type":"ContainerStarted","Data":"980c37f4060e13ed6c4f52d01bf60d00052eda944747b0fa9e5898365c338062"}
Oct 30 22:28:48.524897 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:28:48.524521    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-9513bb-58f7d87d8d-7bcdq"
Oct 30 22:28:48.642731 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:28:48.642673    4473 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-9513bb-58f7d87d8d-7bcdq" podStartSLOduration=2.844590153 podStartE2EDuration="3.642660058s" totalImagesPullingTime="798.069905ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:28:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:28:48.641519698 +0000 UTC m=+2774.414888184" watchObservedRunningTime="2026-10-30 22:28:48.642660058 +0000 UTC m=+2774.416028545"
Oct 30 22:28:50.324052 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:28:50.324015    4473 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fncvg" containerName="extractor"
Oct 30 22:28:54.534882 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:28:54.534852    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-9513bb-58f7d87d8d-7bcdq"
Oct 30 22:28:55.511454 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:28:55.511420    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-single-main-9513bb-97d84485fdzpmr"
Oct 30 22:28:55.511645 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:28:55.511587    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-9513bb-5dcb9zwg2c"
Oct 30 22:29:04.602884 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:29:04.602791    4473 generic.go:356] "Generic (PLEG): container finished" podID="3a63d380-983c-49ae-a8a0-c8e47ead8d1a" containerID="43b8e3bc84ec1dad5a9911989bde0ec6dfde51a28cd71a635605c074420282de" exitCode=0
Oct 30 22:29:04.603333 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:29:04.602872    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-9513bb-openshift-default-67677488d5j5bcz" event={"ID":"3a63d380-983c-49ae-a8a0-c8e47ead8d1a","Type":"ContainerDied","Data":"43b8e3bc84ec1dad5a9911989bde0ec6dfde51a28cd71a635605c074420282de"}
Oct 30 22:29:04.603378 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:29:04.603356    4473 scope.go:122] "RemoveContainer" containerID="43b8e3bc84ec1dad5a9911989bde0ec6dfde51a28cd71a635605c074420282de"
Oct 30 22:29:04.604744 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:29:04.604713    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:29:04.604807 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:29:04.604793    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:29:04.604846 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:29:04.604828    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:29:05.610001 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:29:05.609959    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-9513bb-openshift-default-67677488d5j5bcz" event={"ID":"3a63d380-983c-49ae-a8a0-c8e47ead8d1a","Type":"ContainerStarted","Data":"ddb2dadf96d04b3bb8622df2882384f19b257627e92bdb507436b3df2937617c"}
Oct 30 22:29:06.065625 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:29:06.065587    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-single-main-9513bb-openshift-default-67677488d5j5bcz"
Oct 30 22:29:06.066895 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:29:06.066871    4473 patch_prober.go:29] interesting pod/e2e-worker-single-main-9513bb-openshift-default-67677488d5j5bcz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.150:15021/healthz/ready\": dial tcp 10.133.0.150:15021: connect: connection refused" start-of-body=
Oct 30 22:29:06.067000 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:29:06.066948    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-9513bb-openshift-default-67677488d5j5bcz" podUID="3a63d380-983c-49ae-a8a0-c8e47ead8d1a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.150:15021/healthz/ready\": dial tcp 10.133.0.150:15021: connect: connection refused"
Oct 30 22:29:07.065870 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:29:07.065831    4473 patch_prober.go:29] interesting pod/e2e-worker-single-main-9513bb-openshift-default-67677488d5j5bcz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.150:15021/healthz/ready\": dial tcp 10.133.0.150:15021: connect: connection refused" start-of-body=
Oct 30 22:29:07.066312 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:29:07.065890    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-9513bb-openshift-default-67677488d5j5bcz" podUID="3a63d380-983c-49ae-a8a0-c8e47ead8d1a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.150:15021/healthz/ready\": dial tcp 10.133.0.150:15021: connect: connection refused"
Oct 30 22:29:07.464829 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:29:07.464795    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-single-main-9513bb-openshift-default-67677488d5j5bcz"
Oct 30 22:29:08.065728 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:29:08.065694    4473 patch_prober.go:29] interesting pod/e2e-worker-single-main-9513bb-openshift-default-67677488d5j5bcz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.150:15021/healthz/ready\": dial tcp 10.133.0.150:15021: connect: connection refused" start-of-body=
Oct 30 22:29:08.065912 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:29:08.065750    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-9513bb-openshift-default-67677488d5j5bcz" podUID="3a63d380-983c-49ae-a8a0-c8e47ead8d1a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.150:15021/healthz/ready\": dial tcp 10.133.0.150:15021: connect: connection refused"
Oct 30 22:29:08.890654 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:29:08.890612    4473 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-single-main-9513bb/simulator-single-main-9513bb-kserve-85d5d45c44-pzdxp"]
Oct 30 22:29:08.894289 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:29:08.894266    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-single-main-9513bb/simulator-single-main-9513bb-kserve-85d5d45c44-pzdxp"
Oct 30 22:29:08.994788 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:29:08.994748    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aaa0bfc0-fcc5-42f6-b193-045605590a23-home\") pod \"simulator-single-main-9513bb-kserve-85d5d45c44-pzdxp\" (UID: \"aaa0bfc0-fcc5-42f6-b193-045605590a23\") " pod="e2e-models-e2e-worker-single-main-9513bb/simulator-single-main-9513bb-kserve-85d5d45c44-pzdxp"
Oct 30 22:29:08.995010 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:29:08.994798    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aaa0bfc0-fcc5-42f6-b193-045605590a23-tls-certs\") pod \"simulator-single-main-9513bb-kserve-85d5d45c44-pzdxp\" (UID: \"aaa0bfc0-fcc5-42f6-b193-045605590a23\") " pod="e2e-models-e2e-worker-single-main-9513bb/simulator-single-main-9513bb-kserve-85d5d45c44-pzdxp"
Oct 30 22:29:08.995010 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:29:08.994873    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-86vv9\" (UniqueName: \"kubernetes.io/projected/aaa0bfc0-fcc5-42f6-b193-045605590a23-kube-api-access-86vv9\") pod \"simulator-single-main-9513bb-kserve-85d5d45c44-pzdxp\" (UID: \"aaa0bfc0-fcc5-42f6-b193-045605590a23\") " pod="e2e-models-e2e-worker-single-main-9513bb/simulator-single-main-9513bb-kserve-85d5d45c44-pzdxp"
Oct 30 22:29:08.995010 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:29:08.994941    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aaa0bfc0-fcc5-42f6-b193-045605590a23-dshm\") pod \"simulator-single-main-9513bb-kserve-85d5d45c44-pzdxp\" (UID: \"aaa0bfc0-fcc5-42f6-b193-045605590a23\") " pod="e2e-models-e2e-worker-single-main-9513bb/simulator-single-main-9513bb-kserve-85d5d45c44-pzdxp"
Oct 30 22:29:08.995199 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:29:08.995073    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aaa0bfc0-fcc5-42f6-b193-045605590a23-tmp-dir\") pod \"simulator-single-main-9513bb-kserve-85d5d45c44-pzdxp\" (UID: \"aaa0bfc0-fcc5-42f6-b193-045605590a23\") " pod="e2e-models-e2e-worker-single-main-9513bb/simulator-single-main-9513bb-kserve-85d5d45c44-pzdxp"
Oct 30 22:29:08.995253 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:29:08.995227    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aaa0bfc0-fcc5-42f6-b193-045605590a23-model-cache\") pod \"simulator-single-main-9513bb-kserve-85d5d45c44-pzdxp\" (UID: \"aaa0bfc0-fcc5-42f6-b193-045605590a23\") " pod="e2e-models-e2e-worker-single-main-9513bb/simulator-single-main-9513bb-kserve-85d5d45c44-pzdxp"
Oct 30 22:29:09.000617 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:29:09.000594    4473 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-single-main-9513bb\"/\"openshift-service-ca.crt\""
Oct 30 22:29:09.000720 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:29:09.000662    4473 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-single-main-9513bb\"/\"kube-root-ca.crt\""
Oct 30 22:29:09.000720 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:29:09.000665    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-single-main-9513bb\"/\"default-dockercfg-hctqz\""
Oct 30 22:29:09.000862 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:29:09.000844    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-single-main-9513bb\"/\"simulator-single-main-9513bb-kserve-self-signed-certs\""
Oct 30 22:29:09.002820 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:29:09.002799    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-single-main-9513bb/simulator-single-main-9513bb-kserve-85d5d45c44-pzdxp"]
Oct 30 22:29:09.065534 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:29:09.065503    4473 patch_prober.go:29] interesting pod/e2e-worker-single-main-9513bb-openshift-default-67677488d5j5bcz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.150:15021/healthz/ready\": dial tcp 10.133.0.150:15021: connect: connection refused" start-of-body=
Oct 30 22:29:09.065720 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:29:09.065566    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-9513bb-openshift-default-67677488d5j5bcz" podUID="3a63d380-983c-49ae-a8a0-c8e47ead8d1a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.150:15021/healthz/ready\": dial tcp 10.133.0.150:15021: connect: connection refused"
Oct 30 22:29:09.096621 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:29:09.096579    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aaa0bfc0-fcc5-42f6-b193-045605590a23-home\") pod \"simulator-single-main-9513bb-kserve-85d5d45c44-pzdxp\" (UID: \"aaa0bfc0-fcc5-42f6-b193-045605590a23\") " pod="e2e-models-e2e-worker-single-main-9513bb/simulator-single-main-9513bb-kserve-85d5d45c44-pzdxp"
Oct 30 22:29:09.097142 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:29:09.096626    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aaa0bfc0-fcc5-42f6-b193-045605590a23-tls-certs\") pod \"simulator-single-main-9513bb-kserve-85d5d45c44-pzdxp\" (UID: \"aaa0bfc0-fcc5-42f6-b193-045605590a23\") " pod="e2e-models-e2e-worker-single-main-9513bb/simulator-single-main-9513bb-kserve-85d5d45c44-pzdxp"
Oct 30 22:29:09.097142 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:29:09.096752    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-86vv9\" (UniqueName: \"kubernetes.io/projected/aaa0bfc0-fcc5-42f6-b193-045605590a23-kube-api-access-86vv9\") pod \"simulator-single-main-9513bb-kserve-85d5d45c44-pzdxp\" (UID: \"aaa0bfc0-fcc5-42f6-b193-045605590a23\") " pod="e2e-models-e2e-worker-single-main-9513bb/simulator-single-main-9513bb-kserve-85d5d45c44-pzdxp"
Oct 30 22:29:09.097142 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:29:09.096791    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aaa0bfc0-fcc5-42f6-b193-045605590a23-dshm\") pod \"simulator-single-main-9513bb-kserve-85d5d45c44-pzdxp\" (UID: \"aaa0bfc0-fcc5-42f6-b193-045605590a23\") " pod="e2e-models-e2e-worker-single-main-9513bb/simulator-single-main-9513bb-kserve-85d5d45c44-pzdxp"
Oct 30 22:29:09.097142 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:29:09.096846    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aaa0bfc0-fcc5-42f6-b193-045605590a23-tmp-dir\") pod \"simulator-single-main-9513bb-kserve-85d5d45c44-pzdxp\" (UID: \"aaa0bfc0-fcc5-42f6-b193-045605590a23\") " pod="e2e-models-e2e-worker-single-main-9513bb/simulator-single-main-9513bb-kserve-85d5d45c44-pzdxp"
Oct 30 22:29:09.097142 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:29:09.096990    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aaa0bfc0-fcc5-42f6-b193-045605590a23-model-cache\") pod \"simulator-single-main-9513bb-kserve-85d5d45c44-pzdxp\" (UID: \"aaa0bfc0-fcc5-42f6-b193-045605590a23\") " pod="e2e-models-e2e-worker-single-main-9513bb/simulator-single-main-9513bb-kserve-85d5d45c44-pzdxp"
Oct 30 22:29:09.097142 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:29:09.097009    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aaa0bfc0-fcc5-42f6-b193-045605590a23-home\") pod \"simulator-single-main-9513bb-kserve-85d5d45c44-pzdxp\" (UID: \"aaa0bfc0-fcc5-42f6-b193-045605590a23\") " pod="e2e-models-e2e-worker-single-main-9513bb/simulator-single-main-9513bb-kserve-85d5d45c44-pzdxp"
Oct 30 22:29:09.097372 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:29:09.097252    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aaa0bfc0-fcc5-42f6-b193-045605590a23-tmp-dir\") pod \"simulator-single-main-9513bb-kserve-85d5d45c44-pzdxp\" (UID: \"aaa0bfc0-fcc5-42f6-b193-045605590a23\") " pod="e2e-models-e2e-worker-single-main-9513bb/simulator-single-main-9513bb-kserve-85d5d45c44-pzdxp"
Oct 30 22:29:09.097410 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:29:09.097368    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aaa0bfc0-fcc5-42f6-b193-045605590a23-model-cache\") pod \"simulator-single-main-9513bb-kserve-85d5d45c44-pzdxp\" (UID: \"aaa0bfc0-fcc5-42f6-b193-045605590a23\") " pod="e2e-models-e2e-worker-single-main-9513bb/simulator-single-main-9513bb-kserve-85d5d45c44-pzdxp"
Oct 30 22:29:09.099234 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:29:09.099216    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aaa0bfc0-fcc5-42f6-b193-045605590a23-dshm\") pod \"simulator-single-main-9513bb-kserve-85d5d45c44-pzdxp\" (UID: \"aaa0bfc0-fcc5-42f6-b193-045605590a23\") " pod="e2e-models-e2e-worker-single-main-9513bb/simulator-single-main-9513bb-kserve-85d5d45c44-pzdxp"
Oct 30 22:29:09.099614 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:29:09.099593    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aaa0bfc0-fcc5-42f6-b193-045605590a23-tls-certs\") pod \"simulator-single-main-9513bb-kserve-85d5d45c44-pzdxp\" (UID: \"aaa0bfc0-fcc5-42f6-b193-045605590a23\") " pod="e2e-models-e2e-worker-single-main-9513bb/simulator-single-main-9513bb-kserve-85d5d45c44-pzdxp"
Oct 30 22:29:09.160707 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:29:09.160630    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-86vv9\" (UniqueName: \"kubernetes.io/projected/aaa0bfc0-fcc5-42f6-b193-045605590a23-kube-api-access-86vv9\") pod \"simulator-single-main-9513bb-kserve-85d5d45c44-pzdxp\" (UID: \"aaa0bfc0-fcc5-42f6-b193-045605590a23\") " pod="e2e-models-e2e-worker-single-main-9513bb/simulator-single-main-9513bb-kserve-85d5d45c44-pzdxp"
Oct 30 22:29:09.222121 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:29:09.222080    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-single-main-9513bb/simulator-single-main-9513bb-kserve-85d5d45c44-pzdxp"
Oct 30 22:29:09.637705 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:29:09.637661    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-single-main-9513bb/simulator-single-main-9513bb-kserve-85d5d45c44-pzdxp"]
Oct 30 22:29:09.655126 ip-10-0-140-18 kubenswrapper[4473]: W1030 22:29:09.655077    4473 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podaaa0bfc0_fcc5_42f6_b193_045605590a23.slice/crio-5769be6c835b72d904e14fa7b21e9d34e95de9c92efca68b7c295eb3de2ae2e4 WatchSource:0}: Error finding container 5769be6c835b72d904e14fa7b21e9d34e95de9c92efca68b7c295eb3de2ae2e4: Status 404 returned error can't find the container with id 5769be6c835b72d904e14fa7b21e9d34e95de9c92efca68b7c295eb3de2ae2e4
Oct 30 22:29:10.065444 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:29:10.065348    4473 patch_prober.go:29] interesting pod/e2e-worker-single-main-9513bb-openshift-default-67677488d5j5bcz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.150:15021/healthz/ready\": dial tcp 10.133.0.150:15021: connect: connection refused" start-of-body=
Oct 30 22:29:10.065444 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:29:10.065408    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-9513bb-openshift-default-67677488d5j5bcz" podUID="3a63d380-983c-49ae-a8a0-c8e47ead8d1a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.150:15021/healthz/ready\": dial tcp 10.133.0.150:15021: connect: connection refused"
Oct 30 22:29:10.640272 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:29:10.640227    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-single-main-9513bb/simulator-single-main-9513bb-kserve-85d5d45c44-pzdxp" event={"ID":"aaa0bfc0-fcc5-42f6-b193-045605590a23","Type":"ContainerStarted","Data":"247f85b722a7340e21bbc95d13b9cced100b9e73dc98ed99d9221224bbfcae93"}
Oct 30 22:29:10.640708 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:29:10.640285    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-single-main-9513bb/simulator-single-main-9513bb-kserve-85d5d45c44-pzdxp" event={"ID":"aaa0bfc0-fcc5-42f6-b193-045605590a23","Type":"ContainerStarted","Data":"5769be6c835b72d904e14fa7b21e9d34e95de9c92efca68b7c295eb3de2ae2e4"}
Oct 30 22:29:10.756636 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:29:10.756577    4473 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-single-main-9513bb/simulator-single-main-9513bb-kserve-85d5d45c44-pzdxp" podStartSLOduration=2.756563008 podStartE2EDuration="2.756563008s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:29:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:29:10.755699426 +0000 UTC m=+2796.529067917" watchObservedRunningTime="2026-10-30 22:29:10.756563008 +0000 UTC m=+2796.529931495"
Oct 30 22:29:11.070042 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:29:11.069959    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-single-main-9513bb-openshift-default-67677488d5j5bcz"
Oct 30 22:29:11.071287 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:29:11.071267    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-single-main-9513bb-openshift-default-67677488d5j5bcz"
Oct 30 22:29:11.222718 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:29:11.222677    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-single-main-9513bb/simulator-single-main-9513bb-kserve-85d5d45c44-pzdxp"
Oct 30 22:29:19.223199 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:29:19.223164    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-single-main-9513bb/simulator-single-main-9513bb-kserve-85d5d45c44-pzdxp"
Oct 30 22:29:19.235870 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:29:19.235839    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-single-main-9513bb/simulator-single-main-9513bb-kserve-85d5d45c44-pzdxp"
Oct 30 22:29:19.695887 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:29:19.695857    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-single-main-9513bb/simulator-single-main-9513bb-kserve-85d5d45c44-pzdxp"
Oct 30 22:29:20.325816 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:29:20.325773    4473 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fncvg" containerName="extractor"
Oct 30 22:29:50.324449 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:29:50.324350    4473 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fncvg" containerName="extractor"
Oct 30 22:30:10.982179 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:30:10.982140    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-single-main-9513bb/simulator-single-main-9513bb-kserve-85d5d45c44-pzdxp"]
Oct 30 22:30:17.993372 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:30:17.993334    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-9513bb-97d84485fdzpmr"]
Oct 30 22:30:18.100283 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:30:18.100247    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-9513bb-5dcb9zwg2c"]
Oct 30 22:30:20.323381 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:30:20.323345    4473 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fncvg" containerName="extractor"
Oct 30 22:30:22.994406 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:30:22.994353    4473 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-single-main-9513bb-97d84485fdzpmr" podUID="525bfdc5-b511-42c1-890e-ead9010f1b4b" containerName="payload-processing" containerID="cri-o://4de858a2777cf28178f4b1041949de1e2c4c90158f782b92bb5c66068f43374b" gracePeriod=25
Oct 30 22:30:23.100861 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:30:23.100796    4473 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-9513bb-5dcb9zwg2c" podUID="4d6c8b70-a3dc-42fc-bbfb-3937c3f349cb" containerName="payload-pre-processing" containerID="cri-o://5c6916f1869ac777b433fddc8f1c4826eb28e25d4d429b3dfab5d9e1aa1047ca" gracePeriod=25
Oct 30 22:30:24.051114 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:30:24.051076    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-9513bb-58f7d87d8d-7bcdq"]
Oct 30 22:30:24.051549 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:30:24.051336    4473 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-9513bb-58f7d87d8d-7bcdq" podUID="db69aa0b-8205-4c19-a6c3-838ae5580d0d" containerName="maas-api" containerID="cri-o://980c37f4060e13ed6c4f52d01bf60d00052eda944747b0fa9e5898365c338062" gracePeriod=30
Oct 30 22:30:24.529578 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:30:24.529540    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-9513bb-58f7d87d8d-7bcdq" podUID="db69aa0b-8205-4c19-a6c3-838ae5580d0d" containerName="maas-api" probeResult="failure" output="Get \"https://10.133.0.153:8443/health\": dial tcp 10.133.0.153:8443: connect: connection refused"
Oct 30 22:30:25.312769 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:30:25.312743    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-9513bb-58f7d87d8d-7bcdq"
Oct 30 22:30:25.421375 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:30:25.421341    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/db69aa0b-8205-4c19-a6c3-838ae5580d0d-metrics-certs\" (UniqueName: \"kubernetes.io/secret/db69aa0b-8205-4c19-a6c3-838ae5580d0d-metrics-certs\") pod \"db69aa0b-8205-4c19-a6c3-838ae5580d0d\" (UID: \"db69aa0b-8205-4c19-a6c3-838ae5580d0d\") "
Oct 30 22:30:25.421559 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:30:25.421412    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/db69aa0b-8205-4c19-a6c3-838ae5580d0d-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/db69aa0b-8205-4c19-a6c3-838ae5580d0d-openshift-service-ca\") pod \"db69aa0b-8205-4c19-a6c3-838ae5580d0d\" (UID: \"db69aa0b-8205-4c19-a6c3-838ae5580d0d\") "
Oct 30 22:30:25.421559 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:30:25.421445    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/db69aa0b-8205-4c19-a6c3-838ae5580d0d-kube-api-access-pt5w6\" (UniqueName: \"kubernetes.io/projected/db69aa0b-8205-4c19-a6c3-838ae5580d0d-kube-api-access-pt5w6\") pod \"db69aa0b-8205-4c19-a6c3-838ae5580d0d\" (UID: \"db69aa0b-8205-4c19-a6c3-838ae5580d0d\") "
Oct 30 22:30:25.421559 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:30:25.421510    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/db69aa0b-8205-4c19-a6c3-838ae5580d0d-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/db69aa0b-8205-4c19-a6c3-838ae5580d0d-maas-api-tls\") pod \"db69aa0b-8205-4c19-a6c3-838ae5580d0d\" (UID: \"db69aa0b-8205-4c19-a6c3-838ae5580d0d\") "
Oct 30 22:30:25.421812 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:30:25.421781    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/db69aa0b-8205-4c19-a6c3-838ae5580d0d-openshift-service-ca" pod "db69aa0b-8205-4c19-a6c3-838ae5580d0d" (UID: "db69aa0b-8205-4c19-a6c3-838ae5580d0d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:30:25.421999 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:30:25.421961    4473 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/db69aa0b-8205-4c19-a6c3-838ae5580d0d-openshift-service-ca\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:30:25.423486 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:30:25.423460    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/db69aa0b-8205-4c19-a6c3-838ae5580d0d-metrics-certs" pod "db69aa0b-8205-4c19-a6c3-838ae5580d0d" (UID: "db69aa0b-8205-4c19-a6c3-838ae5580d0d"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:30:25.423628 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:30:25.423606    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/db69aa0b-8205-4c19-a6c3-838ae5580d0d-maas-api-tls" pod "db69aa0b-8205-4c19-a6c3-838ae5580d0d" (UID: "db69aa0b-8205-4c19-a6c3-838ae5580d0d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:30:25.423767 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:30:25.423750    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/db69aa0b-8205-4c19-a6c3-838ae5580d0d-kube-api-access-pt5w6" pod "db69aa0b-8205-4c19-a6c3-838ae5580d0d" (UID: "db69aa0b-8205-4c19-a6c3-838ae5580d0d"). InnerVolumeSpecName "kube-api-access-pt5w6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:30:25.509301 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:30:25.509256    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-9513bb-5dcb9zwg2c" podUID="4d6c8b70-a3dc-42fc-bbfb-3937c3f349cb" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:30:25.509514 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:30:25.509273    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-single-main-9513bb-97d84485fdzpmr" podUID="525bfdc5-b511-42c1-890e-ead9010f1b4b" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:30:25.523534 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:30:25.523505    4473 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/db69aa0b-8205-4c19-a6c3-838ae5580d0d-metrics-certs\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:30:25.523534 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:30:25.523532    4473 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pt5w6\" (UniqueName: \"kubernetes.io/projected/db69aa0b-8205-4c19-a6c3-838ae5580d0d-kube-api-access-pt5w6\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:30:25.523718 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:30:25.523544    4473 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/db69aa0b-8205-4c19-a6c3-838ae5580d0d-maas-api-tls\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:30:26.003330 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:30:26.003276    4473 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-single-main-9513bb/simulator-single-main-9513bb-kserve-85d5d45c44-pzdxp" podUID="aaa0bfc0-fcc5-42f6-b193-045605590a23" containerName="main" containerID="cri-o://247f85b722a7340e21bbc95d13b9cced100b9e73dc98ed99d9221224bbfcae93" gracePeriod=45
Oct 30 22:30:26.020492 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:30:26.020464    4473 generic.go:356] "Generic (PLEG): container finished" podID="db69aa0b-8205-4c19-a6c3-838ae5580d0d" containerID="980c37f4060e13ed6c4f52d01bf60d00052eda944747b0fa9e5898365c338062" exitCode=0
Oct 30 22:30:26.020634 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:30:26.020507    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-9513bb-58f7d87d8d-7bcdq"
Oct 30 22:30:26.020634 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:30:26.020543    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-9513bb-58f7d87d8d-7bcdq" event={"ID":"db69aa0b-8205-4c19-a6c3-838ae5580d0d","Type":"ContainerDied","Data":"980c37f4060e13ed6c4f52d01bf60d00052eda944747b0fa9e5898365c338062"}
Oct 30 22:30:26.020634 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:30:26.020596    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-9513bb-58f7d87d8d-7bcdq" event={"ID":"db69aa0b-8205-4c19-a6c3-838ae5580d0d","Type":"ContainerDied","Data":"4dffd20a5e08979da97c07f4f8384d5ee33b72efa4e697c35ce9a0df617dbbc8"}
Oct 30 22:30:26.020634 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:30:26.020610    4473 scope.go:122] "RemoveContainer" containerID="980c37f4060e13ed6c4f52d01bf60d00052eda944747b0fa9e5898365c338062"
Oct 30 22:30:26.114848 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:30:26.114822    4473 scope.go:122] "RemoveContainer" containerID="980c37f4060e13ed6c4f52d01bf60d00052eda944747b0fa9e5898365c338062"
Oct 30 22:30:26.115253 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:30:26.115222    4473 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"980c37f4060e13ed6c4f52d01bf60d00052eda944747b0fa9e5898365c338062\": container with ID starting with 980c37f4060e13ed6c4f52d01bf60d00052eda944747b0fa9e5898365c338062 not found: ID does not exist" containerID="980c37f4060e13ed6c4f52d01bf60d00052eda944747b0fa9e5898365c338062"
Oct 30 22:30:26.115336 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:30:26.115267    4473 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"980c37f4060e13ed6c4f52d01bf60d00052eda944747b0fa9e5898365c338062"} err="failed to get container status \"980c37f4060e13ed6c4f52d01bf60d00052eda944747b0fa9e5898365c338062\": rpc error: code = NotFound desc = could not find container \"980c37f4060e13ed6c4f52d01bf60d00052eda944747b0fa9e5898365c338062\": container with ID starting with 980c37f4060e13ed6c4f52d01bf60d00052eda944747b0fa9e5898365c338062 not found: ID does not exist"
Oct 30 22:30:26.191720 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:30:26.191684    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-9513bb-58f7d87d8d-7bcdq"]
Oct 30 22:30:26.194076 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:30:26.194047    4473 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-9513bb-58f7d87d8d-7bcdq"]
Oct 30 22:30:26.255573 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:30:26.255491    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-single-main-9513bb/simulator-single-main-9513bb-kserve-85d5d45c44-pzdxp"
Oct 30 22:30:26.331641 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:30:26.331610    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/aaa0bfc0-fcc5-42f6-b193-045605590a23-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aaa0bfc0-fcc5-42f6-b193-045605590a23-tmp-dir\") pod \"aaa0bfc0-fcc5-42f6-b193-045605590a23\" (UID: \"aaa0bfc0-fcc5-42f6-b193-045605590a23\") "
Oct 30 22:30:26.332129 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:30:26.331674    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/aaa0bfc0-fcc5-42f6-b193-045605590a23-kube-api-access-86vv9\" (UniqueName: \"kubernetes.io/projected/aaa0bfc0-fcc5-42f6-b193-045605590a23-kube-api-access-86vv9\") pod \"aaa0bfc0-fcc5-42f6-b193-045605590a23\" (UID: \"aaa0bfc0-fcc5-42f6-b193-045605590a23\") "
Oct 30 22:30:26.332129 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:30:26.331729    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/aaa0bfc0-fcc5-42f6-b193-045605590a23-home\" (UniqueName: \"kubernetes.io/empty-dir/aaa0bfc0-fcc5-42f6-b193-045605590a23-home\") pod \"aaa0bfc0-fcc5-42f6-b193-045605590a23\" (UID: \"aaa0bfc0-fcc5-42f6-b193-045605590a23\") "
Oct 30 22:30:26.332129 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:30:26.331786    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/aaa0bfc0-fcc5-42f6-b193-045605590a23-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aaa0bfc0-fcc5-42f6-b193-045605590a23-model-cache\") pod \"aaa0bfc0-fcc5-42f6-b193-045605590a23\" (UID: \"aaa0bfc0-fcc5-42f6-b193-045605590a23\") "
Oct 30 22:30:26.332129 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:30:26.331865    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/aaa0bfc0-fcc5-42f6-b193-045605590a23-tls-certs\" (UniqueName: \"kubernetes.io/secret/aaa0bfc0-fcc5-42f6-b193-045605590a23-tls-certs\") pod \"aaa0bfc0-fcc5-42f6-b193-045605590a23\" (UID: \"aaa0bfc0-fcc5-42f6-b193-045605590a23\") "
Oct 30 22:30:26.332129 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:30:26.331938    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/aaa0bfc0-fcc5-42f6-b193-045605590a23-dshm\" (UniqueName: \"kubernetes.io/empty-dir/aaa0bfc0-fcc5-42f6-b193-045605590a23-dshm\") pod \"aaa0bfc0-fcc5-42f6-b193-045605590a23\" (UID: \"aaa0bfc0-fcc5-42f6-b193-045605590a23\") "
Oct 30 22:30:26.332129 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:30:26.331905    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aaa0bfc0-fcc5-42f6-b193-045605590a23-tmp-dir" pod "aaa0bfc0-fcc5-42f6-b193-045605590a23" (UID: "aaa0bfc0-fcc5-42f6-b193-045605590a23"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:30:26.332129 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:30:26.332003    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aaa0bfc0-fcc5-42f6-b193-045605590a23-home" pod "aaa0bfc0-fcc5-42f6-b193-045605590a23" (UID: "aaa0bfc0-fcc5-42f6-b193-045605590a23"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:30:26.332129 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:30:26.332073    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aaa0bfc0-fcc5-42f6-b193-045605590a23-model-cache" pod "aaa0bfc0-fcc5-42f6-b193-045605590a23" (UID: "aaa0bfc0-fcc5-42f6-b193-045605590a23"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:30:26.332417 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:30:26.332275    4473 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aaa0bfc0-fcc5-42f6-b193-045605590a23-model-cache\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:30:26.332417 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:30:26.332308    4473 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aaa0bfc0-fcc5-42f6-b193-045605590a23-tmp-dir\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:30:26.332417 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:30:26.332327    4473 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aaa0bfc0-fcc5-42f6-b193-045605590a23-home\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:30:26.334142 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:30:26.334118    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aaa0bfc0-fcc5-42f6-b193-045605590a23-tls-certs" pod "aaa0bfc0-fcc5-42f6-b193-045605590a23" (UID: "aaa0bfc0-fcc5-42f6-b193-045605590a23"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:30:26.334244 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:30:26.334141    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aaa0bfc0-fcc5-42f6-b193-045605590a23-kube-api-access-86vv9" pod "aaa0bfc0-fcc5-42f6-b193-045605590a23" (UID: "aaa0bfc0-fcc5-42f6-b193-045605590a23"). InnerVolumeSpecName "kube-api-access-86vv9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:30:26.334244 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:30:26.334163    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aaa0bfc0-fcc5-42f6-b193-045605590a23-dshm" pod "aaa0bfc0-fcc5-42f6-b193-045605590a23" (UID: "aaa0bfc0-fcc5-42f6-b193-045605590a23"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:30:26.356441 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:30:26.356413    4473 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="db69aa0b-8205-4c19-a6c3-838ae5580d0d" path="/var/lib/kubelet/pods/db69aa0b-8205-4c19-a6c3-838ae5580d0d/volumes"
Oct 30 22:30:26.434114 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:30:26.434079    4473 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aaa0bfc0-fcc5-42f6-b193-045605590a23-dshm\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:30:26.434114 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:30:26.434114    4473 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-86vv9\" (UniqueName: \"kubernetes.io/projected/aaa0bfc0-fcc5-42f6-b193-045605590a23-kube-api-access-86vv9\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:30:26.434321 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:30:26.434130    4473 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aaa0bfc0-fcc5-42f6-b193-045605590a23-tls-certs\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:30:27.027904 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:30:27.027867    4473 generic.go:356] "Generic (PLEG): container finished" podID="aaa0bfc0-fcc5-42f6-b193-045605590a23" containerID="247f85b722a7340e21bbc95d13b9cced100b9e73dc98ed99d9221224bbfcae93" exitCode=0
Oct 30 22:30:27.028128 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:30:27.027949    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-single-main-9513bb/simulator-single-main-9513bb-kserve-85d5d45c44-pzdxp" event={"ID":"aaa0bfc0-fcc5-42f6-b193-045605590a23","Type":"ContainerDied","Data":"247f85b722a7340e21bbc95d13b9cced100b9e73dc98ed99d9221224bbfcae93"}
Oct 30 22:30:27.028128 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:30:27.027957    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-single-main-9513bb/simulator-single-main-9513bb-kserve-85d5d45c44-pzdxp"
Oct 30 22:30:27.028128 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:30:27.027989    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-single-main-9513bb/simulator-single-main-9513bb-kserve-85d5d45c44-pzdxp" event={"ID":"aaa0bfc0-fcc5-42f6-b193-045605590a23","Type":"ContainerDied","Data":"5769be6c835b72d904e14fa7b21e9d34e95de9c92efca68b7c295eb3de2ae2e4"}
Oct 30 22:30:27.028128 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:30:27.028007    4473 scope.go:122] "RemoveContainer" containerID="247f85b722a7340e21bbc95d13b9cced100b9e73dc98ed99d9221224bbfcae93"
Oct 30 22:30:27.036764 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:30:27.036738    4473 scope.go:122] "RemoveContainer" containerID="247f85b722a7340e21bbc95d13b9cced100b9e73dc98ed99d9221224bbfcae93"
Oct 30 22:30:27.037164 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:30:27.037144    4473 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"247f85b722a7340e21bbc95d13b9cced100b9e73dc98ed99d9221224bbfcae93\": container with ID starting with 247f85b722a7340e21bbc95d13b9cced100b9e73dc98ed99d9221224bbfcae93 not found: ID does not exist" containerID="247f85b722a7340e21bbc95d13b9cced100b9e73dc98ed99d9221224bbfcae93"
Oct 30 22:30:27.037222 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:30:27.037172    4473 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"247f85b722a7340e21bbc95d13b9cced100b9e73dc98ed99d9221224bbfcae93"} err="failed to get container status \"247f85b722a7340e21bbc95d13b9cced100b9e73dc98ed99d9221224bbfcae93\": rpc error: code = NotFound desc = could not find container \"247f85b722a7340e21bbc95d13b9cced100b9e73dc98ed99d9221224bbfcae93\": container with ID starting with 247f85b722a7340e21bbc95d13b9cced100b9e73dc98ed99d9221224bbfcae93 not found: ID does not exist"
Oct 30 22:30:27.194657 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:30:27.194623    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-single-main-9513bb/simulator-single-main-9513bb-kserve-85d5d45c44-pzdxp"]
Oct 30 22:30:27.244958 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:30:27.244914    4473 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-single-main-9513bb/simulator-single-main-9513bb-kserve-85d5d45c44-pzdxp"]
Oct 30 22:30:28.355205 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:30:28.355171    4473 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="aaa0bfc0-fcc5-42f6-b193-045605590a23" path="/var/lib/kubelet/pods/aaa0bfc0-fcc5-42f6-b193-045605590a23/volumes"
Oct 30 22:30:32.907140 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:30:32.907099    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-single-main-9513bb-openshift-default-67677488d5j5bcz"]
Oct 30 22:30:32.907635 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:30:32.907336    4473 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-single-main-9513bb-openshift-default-67677488d5j5bcz" podUID="3a63d380-983c-49ae-a8a0-c8e47ead8d1a" containerName="istio-proxy" containerID="cri-o://ddb2dadf96d04b3bb8622df2882384f19b257627e92bdb507436b3df2937617c" gracePeriod=30
Oct 30 22:30:35.509445 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:30:35.509409    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-single-main-9513bb-97d84485fdzpmr" podUID="525bfdc5-b511-42c1-890e-ead9010f1b4b" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:30:35.509837 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:30:35.509409    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-9513bb-5dcb9zwg2c" podUID="4d6c8b70-a3dc-42fc-bbfb-3937c3f349cb" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:30:38.094153 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:30:38.094118    4473 generic.go:356] "Generic (PLEG): container finished" podID="3a63d380-983c-49ae-a8a0-c8e47ead8d1a" containerID="ddb2dadf96d04b3bb8622df2882384f19b257627e92bdb507436b3df2937617c" exitCode=0
Oct 30 22:30:38.094574 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:30:38.094184    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-9513bb-openshift-default-67677488d5j5bcz" event={"ID":"3a63d380-983c-49ae-a8a0-c8e47ead8d1a","Type":"ContainerDied","Data":"ddb2dadf96d04b3bb8622df2882384f19b257627e92bdb507436b3df2937617c"}
Oct 30 22:30:38.094574 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:30:38.094248    4473 scope.go:122] "RemoveContainer" containerID="43b8e3bc84ec1dad5a9911989bde0ec6dfde51a28cd71a635605c074420282de"
Oct 30 22:30:38.157762 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:30:38.157733    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-single-main-9513bb-openshift-default-67677488d5j5bcz"
Oct 30 22:30:38.259944 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:30:38.259834    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3a63d380-983c-49ae-a8a0-c8e47ead8d1a-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a63d380-983c-49ae-a8a0-c8e47ead8d1a-credential-socket\") pod \"3a63d380-983c-49ae-a8a0-c8e47ead8d1a\" (UID: \"3a63d380-983c-49ae-a8a0-c8e47ead8d1a\") "
Oct 30 22:30:38.259944 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:30:38.259874    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3a63d380-983c-49ae-a8a0-c8e47ead8d1a-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3a63d380-983c-49ae-a8a0-c8e47ead8d1a-workload-certs\") pod \"3a63d380-983c-49ae-a8a0-c8e47ead8d1a\" (UID: \"3a63d380-983c-49ae-a8a0-c8e47ead8d1a\") "
Oct 30 22:30:38.259944 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:30:38.259911    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/3a63d380-983c-49ae-a8a0-c8e47ead8d1a-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3a63d380-983c-49ae-a8a0-c8e47ead8d1a-istio-podinfo\") pod \"3a63d380-983c-49ae-a8a0-c8e47ead8d1a\" (UID: \"3a63d380-983c-49ae-a8a0-c8e47ead8d1a\") "
Oct 30 22:30:38.260293 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:30:38.259995    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3a63d380-983c-49ae-a8a0-c8e47ead8d1a-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3a63d380-983c-49ae-a8a0-c8e47ead8d1a-istio-envoy\") pod \"3a63d380-983c-49ae-a8a0-c8e47ead8d1a\" (UID: \"3a63d380-983c-49ae-a8a0-c8e47ead8d1a\") "
Oct 30 22:30:38.260293 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:30:38.260015    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3a63d380-983c-49ae-a8a0-c8e47ead8d1a-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3a63d380-983c-49ae-a8a0-c8e47ead8d1a-istio-data\") pod \"3a63d380-983c-49ae-a8a0-c8e47ead8d1a\" (UID: \"3a63d380-983c-49ae-a8a0-c8e47ead8d1a\") "
Oct 30 22:30:38.260293 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:30:38.260037    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3a63d380-983c-49ae-a8a0-c8e47ead8d1a-kube-api-access-w2m7r\" (UniqueName: \"kubernetes.io/projected/3a63d380-983c-49ae-a8a0-c8e47ead8d1a-kube-api-access-w2m7r\") pod \"3a63d380-983c-49ae-a8a0-c8e47ead8d1a\" (UID: \"3a63d380-983c-49ae-a8a0-c8e47ead8d1a\") "
Oct 30 22:30:38.260293 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:30:38.260064    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3a63d380-983c-49ae-a8a0-c8e47ead8d1a-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a63d380-983c-49ae-a8a0-c8e47ead8d1a-workload-socket\") pod \"3a63d380-983c-49ae-a8a0-c8e47ead8d1a\" (UID: \"3a63d380-983c-49ae-a8a0-c8e47ead8d1a\") "
Oct 30 22:30:38.260293 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:30:38.260110    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3a63d380-983c-49ae-a8a0-c8e47ead8d1a-istio-token\" (UniqueName: \"kubernetes.io/projected/3a63d380-983c-49ae-a8a0-c8e47ead8d1a-istio-token\") pod \"3a63d380-983c-49ae-a8a0-c8e47ead8d1a\" (UID: \"3a63d380-983c-49ae-a8a0-c8e47ead8d1a\") "
Oct 30 22:30:38.260293 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:30:38.260150    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3a63d380-983c-49ae-a8a0-c8e47ead8d1a-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3a63d380-983c-49ae-a8a0-c8e47ead8d1a-istiod-ca-cert\") pod \"3a63d380-983c-49ae-a8a0-c8e47ead8d1a\" (UID: \"3a63d380-983c-49ae-a8a0-c8e47ead8d1a\") "
Oct 30 22:30:38.260293 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:30:38.260210    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3a63d380-983c-49ae-a8a0-c8e47ead8d1a-credential-socket" pod "3a63d380-983c-49ae-a8a0-c8e47ead8d1a" (UID: "3a63d380-983c-49ae-a8a0-c8e47ead8d1a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:30:38.260293 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:30:38.260223    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3a63d380-983c-49ae-a8a0-c8e47ead8d1a-workload-certs" pod "3a63d380-983c-49ae-a8a0-c8e47ead8d1a" (UID: "3a63d380-983c-49ae-a8a0-c8e47ead8d1a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:30:38.260735 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:30:38.260385    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3a63d380-983c-49ae-a8a0-c8e47ead8d1a-istio-data" pod "3a63d380-983c-49ae-a8a0-c8e47ead8d1a" (UID: "3a63d380-983c-49ae-a8a0-c8e47ead8d1a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:30:38.260735 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:30:38.260533    4473 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a63d380-983c-49ae-a8a0-c8e47ead8d1a-credential-socket\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:30:38.260735 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:30:38.260560    4473 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3a63d380-983c-49ae-a8a0-c8e47ead8d1a-workload-certs\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:30:38.260735 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:30:38.260577    4473 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3a63d380-983c-49ae-a8a0-c8e47ead8d1a-istio-data\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:30:38.260735 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:30:38.260605    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3a63d380-983c-49ae-a8a0-c8e47ead8d1a-workload-socket" pod "3a63d380-983c-49ae-a8a0-c8e47ead8d1a" (UID: "3a63d380-983c-49ae-a8a0-c8e47ead8d1a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:30:38.260735 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:30:38.260624    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3a63d380-983c-49ae-a8a0-c8e47ead8d1a-istiod-ca-cert" pod "3a63d380-983c-49ae-a8a0-c8e47ead8d1a" (UID: "3a63d380-983c-49ae-a8a0-c8e47ead8d1a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:30:38.262399 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:30:38.262379    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3a63d380-983c-49ae-a8a0-c8e47ead8d1a-istio-podinfo" pod "3a63d380-983c-49ae-a8a0-c8e47ead8d1a" (UID: "3a63d380-983c-49ae-a8a0-c8e47ead8d1a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:30:38.262503 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:30:38.262400    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3a63d380-983c-49ae-a8a0-c8e47ead8d1a-istio-token" pod "3a63d380-983c-49ae-a8a0-c8e47ead8d1a" (UID: "3a63d380-983c-49ae-a8a0-c8e47ead8d1a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:30:38.262910 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:30:38.262892    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3a63d380-983c-49ae-a8a0-c8e47ead8d1a-istio-envoy" pod "3a63d380-983c-49ae-a8a0-c8e47ead8d1a" (UID: "3a63d380-983c-49ae-a8a0-c8e47ead8d1a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:30:38.262998 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:30:38.262910    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3a63d380-983c-49ae-a8a0-c8e47ead8d1a-kube-api-access-w2m7r" pod "3a63d380-983c-49ae-a8a0-c8e47ead8d1a" (UID: "3a63d380-983c-49ae-a8a0-c8e47ead8d1a"). InnerVolumeSpecName "kube-api-access-w2m7r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:30:38.361479 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:30:38.361445    4473 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3a63d380-983c-49ae-a8a0-c8e47ead8d1a-istiod-ca-cert\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:30:38.361479 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:30:38.361480    4473 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3a63d380-983c-49ae-a8a0-c8e47ead8d1a-istio-podinfo\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:30:38.361735 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:30:38.361490    4473 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3a63d380-983c-49ae-a8a0-c8e47ead8d1a-istio-envoy\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:30:38.361735 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:30:38.361498    4473 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w2m7r\" (UniqueName: \"kubernetes.io/projected/3a63d380-983c-49ae-a8a0-c8e47ead8d1a-kube-api-access-w2m7r\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:30:38.361735 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:30:38.361508    4473 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a63d380-983c-49ae-a8a0-c8e47ead8d1a-workload-socket\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:30:38.361735 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:30:38.361517    4473 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3a63d380-983c-49ae-a8a0-c8e47ead8d1a-istio-token\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:30:39.100902 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:30:39.100869    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-9513bb-openshift-default-67677488d5j5bcz" event={"ID":"3a63d380-983c-49ae-a8a0-c8e47ead8d1a","Type":"ContainerDied","Data":"1393701dcdb78fd5a8785d9a1f8e97c37aa7cc2c01323cea529751493dae53e8"}
Oct 30 22:30:39.101333 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:30:39.100938    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-single-main-9513bb-openshift-default-67677488d5j5bcz"
Oct 30 22:30:39.101333 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:30:39.100945    4473 scope.go:122] "RemoveContainer" containerID="ddb2dadf96d04b3bb8622df2882384f19b257627e92bdb507436b3df2937617c"
Oct 30 22:30:39.319186 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:30:39.319151    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-single-main-9513bb-openshift-default-67677488d5j5bcz"]
Oct 30 22:30:39.369634 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:30:39.369602    4473 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-single-main-9513bb-openshift-default-67677488d5j5bcz"]
Oct 30 22:30:40.355694 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:30:40.355651    4473 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3a63d380-983c-49ae-a8a0-c8e47ead8d1a" path="/var/lib/kubelet/pods/3a63d380-983c-49ae-a8a0-c8e47ead8d1a/volumes"
Oct 30 22:30:43.123451 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:30:43.123427    4473 generic.go:356] "Generic (PLEG): container finished" podID="525bfdc5-b511-42c1-890e-ead9010f1b4b" containerID="4de858a2777cf28178f4b1041949de1e2c4c90158f782b92bb5c66068f43374b" exitCode=0
Oct 30 22:30:43.123772 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:30:43.123498    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-single-main-9513bb-97d84485fdzpmr" event={"ID":"525bfdc5-b511-42c1-890e-ead9010f1b4b","Type":"ContainerDied","Data":"4de858a2777cf28178f4b1041949de1e2c4c90158f782b92bb5c66068f43374b"}
Oct 30 22:30:43.249692 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:30:43.249658    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-single-main-9513bb-97d84485fdzpmr"
Oct 30 22:30:43.308109 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:30:43.308072    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/525bfdc5-b511-42c1-890e-ead9010f1b4b-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/525bfdc5-b511-42c1-890e-ead9010f1b4b-plugins-config-volume\") pod \"525bfdc5-b511-42c1-890e-ead9010f1b4b\" (UID: \"525bfdc5-b511-42c1-890e-ead9010f1b4b\") "
Oct 30 22:30:43.308314 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:30:43.308176    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/525bfdc5-b511-42c1-890e-ead9010f1b4b-kube-api-access-p4t8j\" (UniqueName: \"kubernetes.io/projected/525bfdc5-b511-42c1-890e-ead9010f1b4b-kube-api-access-p4t8j\") pod \"525bfdc5-b511-42c1-890e-ead9010f1b4b\" (UID: \"525bfdc5-b511-42c1-890e-ead9010f1b4b\") "
Oct 30 22:30:43.308482 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:30:43.308453    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/525bfdc5-b511-42c1-890e-ead9010f1b4b-plugins-config-volume" pod "525bfdc5-b511-42c1-890e-ead9010f1b4b" (UID: "525bfdc5-b511-42c1-890e-ead9010f1b4b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:30:43.308627 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:30:43.308612    4473 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/525bfdc5-b511-42c1-890e-ead9010f1b4b-plugins-config-volume\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:30:43.311332 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:30:43.311306    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/525bfdc5-b511-42c1-890e-ead9010f1b4b-kube-api-access-p4t8j" pod "525bfdc5-b511-42c1-890e-ead9010f1b4b" (UID: "525bfdc5-b511-42c1-890e-ead9010f1b4b"). InnerVolumeSpecName "kube-api-access-p4t8j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:30:43.336092 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:30:43.336064    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-9513bb-5dcb9zwg2c"
Oct 30 22:30:43.409258 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:30:43.409215    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4d6c8b70-a3dc-42fc-bbfb-3937c3f349cb-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4d6c8b70-a3dc-42fc-bbfb-3937c3f349cb-plugins-config-volume\") pod \"4d6c8b70-a3dc-42fc-bbfb-3937c3f349cb\" (UID: \"4d6c8b70-a3dc-42fc-bbfb-3937c3f349cb\") "
Oct 30 22:30:43.409258 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:30:43.409264    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4d6c8b70-a3dc-42fc-bbfb-3937c3f349cb-kube-api-access-zdjf9\" (UniqueName: \"kubernetes.io/projected/4d6c8b70-a3dc-42fc-bbfb-3937c3f349cb-kube-api-access-zdjf9\") pod \"4d6c8b70-a3dc-42fc-bbfb-3937c3f349cb\" (UID: \"4d6c8b70-a3dc-42fc-bbfb-3937c3f349cb\") "
Oct 30 22:30:43.409599 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:30:43.409564    4473 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p4t8j\" (UniqueName: \"kubernetes.io/projected/525bfdc5-b511-42c1-890e-ead9010f1b4b-kube-api-access-p4t8j\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:30:43.409726 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:30:43.409604    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4d6c8b70-a3dc-42fc-bbfb-3937c3f349cb-plugins-config-volume" pod "4d6c8b70-a3dc-42fc-bbfb-3937c3f349cb" (UID: "4d6c8b70-a3dc-42fc-bbfb-3937c3f349cb"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:30:43.411549 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:30:43.411528    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4d6c8b70-a3dc-42fc-bbfb-3937c3f349cb-kube-api-access-zdjf9" pod "4d6c8b70-a3dc-42fc-bbfb-3937c3f349cb" (UID: "4d6c8b70-a3dc-42fc-bbfb-3937c3f349cb"). InnerVolumeSpecName "kube-api-access-zdjf9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:30:43.510276 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:30:43.510226    4473 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4d6c8b70-a3dc-42fc-bbfb-3937c3f349cb-plugins-config-volume\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:30:43.510276 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:30:43.510272    4473 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zdjf9\" (UniqueName: \"kubernetes.io/projected/4d6c8b70-a3dc-42fc-bbfb-3937c3f349cb-kube-api-access-zdjf9\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:30:44.129906 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:30:44.129875    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-single-main-9513bb-97d84485fdzpmr"
Oct 30 22:30:44.130430 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:30:44.129902    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-single-main-9513bb-97d84485fdzpmr" event={"ID":"525bfdc5-b511-42c1-890e-ead9010f1b4b","Type":"ContainerDied","Data":"d19b1a8103479da5d047c72e46a08dddd9e77f9f7551d6aaa25b697f150d7cac"}
Oct 30 22:30:44.130430 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:30:44.130028    4473 scope.go:122] "RemoveContainer" containerID="4de858a2777cf28178f4b1041949de1e2c4c90158f782b92bb5c66068f43374b"
Oct 30 22:30:44.131502 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:30:44.131479    4473 generic.go:356] "Generic (PLEG): container finished" podID="4d6c8b70-a3dc-42fc-bbfb-3937c3f349cb" containerID="5c6916f1869ac777b433fddc8f1c4826eb28e25d4d429b3dfab5d9e1aa1047ca" exitCode=0
Oct 30 22:30:44.131589 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:30:44.131543    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-9513bb-5dcb9zwg2c" event={"ID":"4d6c8b70-a3dc-42fc-bbfb-3937c3f349cb","Type":"ContainerDied","Data":"5c6916f1869ac777b433fddc8f1c4826eb28e25d4d429b3dfab5d9e1aa1047ca"}
Oct 30 22:30:44.131626 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:30:44.131584    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-9513bb-5dcb9zwg2c" event={"ID":"4d6c8b70-a3dc-42fc-bbfb-3937c3f349cb","Type":"ContainerDied","Data":"b9768d35454a64d9df4461e7dab187ab96ff42f9027bdc7cf1ab980d793f1e37"}
Oct 30 22:30:44.131729 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:30:44.131708    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-9513bb-5dcb9zwg2c"
Oct 30 22:30:44.140161 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:30:44.140142    4473 scope.go:122] "RemoveContainer" containerID="5c6916f1869ac777b433fddc8f1c4826eb28e25d4d429b3dfab5d9e1aa1047ca"
Oct 30 22:30:44.148335 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:30:44.148319    4473 scope.go:122] "RemoveContainer" containerID="5c6916f1869ac777b433fddc8f1c4826eb28e25d4d429b3dfab5d9e1aa1047ca"
Oct 30 22:30:44.148629 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:30:44.148608    4473 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5c6916f1869ac777b433fddc8f1c4826eb28e25d4d429b3dfab5d9e1aa1047ca\": container with ID starting with 5c6916f1869ac777b433fddc8f1c4826eb28e25d4d429b3dfab5d9e1aa1047ca not found: ID does not exist" containerID="5c6916f1869ac777b433fddc8f1c4826eb28e25d4d429b3dfab5d9e1aa1047ca"
Oct 30 22:30:44.148675 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:30:44.148639    4473 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5c6916f1869ac777b433fddc8f1c4826eb28e25d4d429b3dfab5d9e1aa1047ca"} err="failed to get container status \"5c6916f1869ac777b433fddc8f1c4826eb28e25d4d429b3dfab5d9e1aa1047ca\": rpc error: code = NotFound desc = could not find container \"5c6916f1869ac777b433fddc8f1c4826eb28e25d4d429b3dfab5d9e1aa1047ca\": container with ID starting with 5c6916f1869ac777b433fddc8f1c4826eb28e25d4d429b3dfab5d9e1aa1047ca not found: ID does not exist"
Oct 30 22:30:44.296467 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:30:44.296382    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-9513bb-5dcb9zwg2c"]
Oct 30 22:30:44.346686 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:30:44.346650    4473 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-9513bb-5dcb9zwg2c"]
Oct 30 22:30:44.355705 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:30:44.355676    4473 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4d6c8b70-a3dc-42fc-bbfb-3937c3f349cb" path="/var/lib/kubelet/pods/4d6c8b70-a3dc-42fc-bbfb-3937c3f349cb/volumes"
Oct 30 22:30:44.409325 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:30:44.409290    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-9513bb-97d84485fdzpmr"]
Oct 30 22:30:44.415624 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:30:44.415596    4473 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-9513bb-97d84485fdzpmr"]
Oct 30 22:30:46.355259 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:30:46.355223    4473 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="525bfdc5-b511-42c1-890e-ead9010f1b4b" path="/var/lib/kubelet/pods/525bfdc5-b511-42c1-890e-ead9010f1b4b/volumes"
Oct 30 22:30:50.323659 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:30:50.323623    4473 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fncvg" containerName="extractor"
Oct 30 22:31:20.324987 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:31:20.324878    4473 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fncvg" containerName="extractor"
Oct 30 22:31:50.324287 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:31:50.324251    4473 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fncvg" containerName="extractor"
Oct 30 22:32:20.324378 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:32:20.324340    4473 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fncvg" containerName="extractor"
Oct 30 22:32:23.579948 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:32:23.579071    4473 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-5b9r4 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.146:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 22:32:23.579948 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:32:23.579141    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-5b9r4" podUID="1557635a-ac42-4385-99f3-879641eb0ccc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.146:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:32:50.325175 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:32:50.325088    4473 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fncvg" containerName="extractor"
Oct 30 22:33:07.025490 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:07.025454    4473 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-single-main-85b9fe-openshift-default-58dc79f89cqfskn"]
Oct 30 22:33:07.026066 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:07.025866    4473 cpu_manager.go:414] "containerMap: removing container" podUID="db69aa0b-8205-4c19-a6c3-838ae5580d0d" containerName="maas-api"
Oct 30 22:33:07.026066 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:07.025877    4473 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="db69aa0b-8205-4c19-a6c3-838ae5580d0d" containerName="maas-api"
Oct 30 22:33:07.026066 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:07.025901    4473 cpu_manager.go:414] "containerMap: removing container" podUID="4d6c8b70-a3dc-42fc-bbfb-3937c3f349cb" containerName="payload-pre-processing"
Oct 30 22:33:07.026066 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:07.025906    4473 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4d6c8b70-a3dc-42fc-bbfb-3937c3f349cb" containerName="payload-pre-processing"
Oct 30 22:33:07.026066 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:07.025936    4473 cpu_manager.go:414] "containerMap: removing container" podUID="525bfdc5-b511-42c1-890e-ead9010f1b4b" containerName="payload-processing"
Oct 30 22:33:07.026066 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:07.025944    4473 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="525bfdc5-b511-42c1-890e-ead9010f1b4b" containerName="payload-processing"
Oct 30 22:33:07.026066 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:07.025956    4473 cpu_manager.go:414] "containerMap: removing container" podUID="3a63d380-983c-49ae-a8a0-c8e47ead8d1a" containerName="istio-proxy"
Oct 30 22:33:07.026066 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:07.025965    4473 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3a63d380-983c-49ae-a8a0-c8e47ead8d1a" containerName="istio-proxy"
Oct 30 22:33:07.026066 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:07.025976    4473 cpu_manager.go:414] "containerMap: removing container" podUID="3a63d380-983c-49ae-a8a0-c8e47ead8d1a" containerName="istio-proxy"
Oct 30 22:33:07.026066 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:07.025980    4473 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3a63d380-983c-49ae-a8a0-c8e47ead8d1a" containerName="istio-proxy"
Oct 30 22:33:07.026066 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:07.025986    4473 cpu_manager.go:414] "containerMap: removing container" podUID="aaa0bfc0-fcc5-42f6-b193-045605590a23" containerName="main"
Oct 30 22:33:07.026066 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:07.025991    4473 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="aaa0bfc0-fcc5-42f6-b193-045605590a23" containerName="main"
Oct 30 22:33:07.026066 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:07.026068    4473 memory_manager.go:365] "RemoveStaleState removing state" podUID="db69aa0b-8205-4c19-a6c3-838ae5580d0d" containerName="maas-api"
Oct 30 22:33:07.026066 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:07.026085    4473 memory_manager.go:365] "RemoveStaleState removing state" podUID="4d6c8b70-a3dc-42fc-bbfb-3937c3f349cb" containerName="payload-pre-processing"
Oct 30 22:33:07.026066 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:07.026095    4473 memory_manager.go:365] "RemoveStaleState removing state" podUID="525bfdc5-b511-42c1-890e-ead9010f1b4b" containerName="payload-processing"
Oct 30 22:33:07.026066 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:07.026104    4473 memory_manager.go:365] "RemoveStaleState removing state" podUID="3a63d380-983c-49ae-a8a0-c8e47ead8d1a" containerName="istio-proxy"
Oct 30 22:33:07.027620 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:07.026114    4473 memory_manager.go:365] "RemoveStaleState removing state" podUID="3a63d380-983c-49ae-a8a0-c8e47ead8d1a" containerName="istio-proxy"
Oct 30 22:33:07.027620 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:07.026123    4473 memory_manager.go:365] "RemoveStaleState removing state" podUID="aaa0bfc0-fcc5-42f6-b193-045605590a23" containerName="main"
Oct 30 22:33:07.029570 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:07.029555    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-single-main-85b9fe-openshift-default-58dc79f89cqfskn"
Oct 30 22:33:07.074362 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:07.074317    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/20409964-6db7-4e50-8369-c031445e2595-istiod-ca-cert\") pod \"e2e-worker-single-main-85b9fe-openshift-default-58dc79f89cqfskn\" (UID: \"20409964-6db7-4e50-8369-c031445e2595\") " pod="openshift-ingress/e2e-worker-single-main-85b9fe-openshift-default-58dc79f89cqfskn"
Oct 30 22:33:07.074643 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:07.074625    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/20409964-6db7-4e50-8369-c031445e2595-istio-token\") pod \"e2e-worker-single-main-85b9fe-openshift-default-58dc79f89cqfskn\" (UID: \"20409964-6db7-4e50-8369-c031445e2595\") " pod="openshift-ingress/e2e-worker-single-main-85b9fe-openshift-default-58dc79f89cqfskn"
Oct 30 22:33:07.074760 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:07.074748    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/20409964-6db7-4e50-8369-c031445e2595-istio-envoy\") pod \"e2e-worker-single-main-85b9fe-openshift-default-58dc79f89cqfskn\" (UID: \"20409964-6db7-4e50-8369-c031445e2595\") " pod="openshift-ingress/e2e-worker-single-main-85b9fe-openshift-default-58dc79f89cqfskn"
Oct 30 22:33:07.074973 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:07.074952    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/20409964-6db7-4e50-8369-c031445e2595-istio-podinfo\") pod \"e2e-worker-single-main-85b9fe-openshift-default-58dc79f89cqfskn\" (UID: \"20409964-6db7-4e50-8369-c031445e2595\") " pod="openshift-ingress/e2e-worker-single-main-85b9fe-openshift-default-58dc79f89cqfskn"
Oct 30 22:33:07.075148 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:07.075123    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/20409964-6db7-4e50-8369-c031445e2595-workload-socket\") pod \"e2e-worker-single-main-85b9fe-openshift-default-58dc79f89cqfskn\" (UID: \"20409964-6db7-4e50-8369-c031445e2595\") " pod="openshift-ingress/e2e-worker-single-main-85b9fe-openshift-default-58dc79f89cqfskn"
Oct 30 22:33:07.075319 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:07.075162    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/20409964-6db7-4e50-8369-c031445e2595-credential-socket\") pod \"e2e-worker-single-main-85b9fe-openshift-default-58dc79f89cqfskn\" (UID: \"20409964-6db7-4e50-8369-c031445e2595\") " pod="openshift-ingress/e2e-worker-single-main-85b9fe-openshift-default-58dc79f89cqfskn"
Oct 30 22:33:07.075319 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:07.075192    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wfng5\" (UniqueName: \"kubernetes.io/projected/20409964-6db7-4e50-8369-c031445e2595-kube-api-access-wfng5\") pod \"e2e-worker-single-main-85b9fe-openshift-default-58dc79f89cqfskn\" (UID: \"20409964-6db7-4e50-8369-c031445e2595\") " pod="openshift-ingress/e2e-worker-single-main-85b9fe-openshift-default-58dc79f89cqfskn"
Oct 30 22:33:07.075319 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:07.075295    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/20409964-6db7-4e50-8369-c031445e2595-workload-certs\") pod \"e2e-worker-single-main-85b9fe-openshift-default-58dc79f89cqfskn\" (UID: \"20409964-6db7-4e50-8369-c031445e2595\") " pod="openshift-ingress/e2e-worker-single-main-85b9fe-openshift-default-58dc79f89cqfskn"
Oct 30 22:33:07.076855 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:07.075321    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/20409964-6db7-4e50-8369-c031445e2595-istio-data\") pod \"e2e-worker-single-main-85b9fe-openshift-default-58dc79f89cqfskn\" (UID: \"20409964-6db7-4e50-8369-c031445e2595\") " pod="openshift-ingress/e2e-worker-single-main-85b9fe-openshift-default-58dc79f89cqfskn"
Oct 30 22:33:07.133162 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:07.133120    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-single-main-85b9fe-openshift-default-dockercfg-47mnp\""
Oct 30 22:33:07.135122 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:07.135081    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-single-main-85b9fe-openshift-default-58dc79f89cqfskn"]
Oct 30 22:33:07.175976 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:07.175933    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/20409964-6db7-4e50-8369-c031445e2595-workload-socket\") pod \"e2e-worker-single-main-85b9fe-openshift-default-58dc79f89cqfskn\" (UID: \"20409964-6db7-4e50-8369-c031445e2595\") " pod="openshift-ingress/e2e-worker-single-main-85b9fe-openshift-default-58dc79f89cqfskn"
Oct 30 22:33:07.176150 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:07.175981    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/20409964-6db7-4e50-8369-c031445e2595-credential-socket\") pod \"e2e-worker-single-main-85b9fe-openshift-default-58dc79f89cqfskn\" (UID: \"20409964-6db7-4e50-8369-c031445e2595\") " pod="openshift-ingress/e2e-worker-single-main-85b9fe-openshift-default-58dc79f89cqfskn"
Oct 30 22:33:07.176150 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:07.176016    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wfng5\" (UniqueName: \"kubernetes.io/projected/20409964-6db7-4e50-8369-c031445e2595-kube-api-access-wfng5\") pod \"e2e-worker-single-main-85b9fe-openshift-default-58dc79f89cqfskn\" (UID: \"20409964-6db7-4e50-8369-c031445e2595\") " pod="openshift-ingress/e2e-worker-single-main-85b9fe-openshift-default-58dc79f89cqfskn"
Oct 30 22:33:07.176150 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:07.176060    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/20409964-6db7-4e50-8369-c031445e2595-workload-certs\") pod \"e2e-worker-single-main-85b9fe-openshift-default-58dc79f89cqfskn\" (UID: \"20409964-6db7-4e50-8369-c031445e2595\") " pod="openshift-ingress/e2e-worker-single-main-85b9fe-openshift-default-58dc79f89cqfskn"
Oct 30 22:33:07.176150 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:07.176085    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/20409964-6db7-4e50-8369-c031445e2595-istio-data\") pod \"e2e-worker-single-main-85b9fe-openshift-default-58dc79f89cqfskn\" (UID: \"20409964-6db7-4e50-8369-c031445e2595\") " pod="openshift-ingress/e2e-worker-single-main-85b9fe-openshift-default-58dc79f89cqfskn"
Oct 30 22:33:07.176379 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:07.176164    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/20409964-6db7-4e50-8369-c031445e2595-istiod-ca-cert\") pod \"e2e-worker-single-main-85b9fe-openshift-default-58dc79f89cqfskn\" (UID: \"20409964-6db7-4e50-8369-c031445e2595\") " pod="openshift-ingress/e2e-worker-single-main-85b9fe-openshift-default-58dc79f89cqfskn"
Oct 30 22:33:07.176379 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:07.176205    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/20409964-6db7-4e50-8369-c031445e2595-istio-token\") pod \"e2e-worker-single-main-85b9fe-openshift-default-58dc79f89cqfskn\" (UID: \"20409964-6db7-4e50-8369-c031445e2595\") " pod="openshift-ingress/e2e-worker-single-main-85b9fe-openshift-default-58dc79f89cqfskn"
Oct 30 22:33:07.176379 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:07.176239    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/20409964-6db7-4e50-8369-c031445e2595-istio-envoy\") pod \"e2e-worker-single-main-85b9fe-openshift-default-58dc79f89cqfskn\" (UID: \"20409964-6db7-4e50-8369-c031445e2595\") " pod="openshift-ingress/e2e-worker-single-main-85b9fe-openshift-default-58dc79f89cqfskn"
Oct 30 22:33:07.176379 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:07.176295    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/20409964-6db7-4e50-8369-c031445e2595-istio-podinfo\") pod \"e2e-worker-single-main-85b9fe-openshift-default-58dc79f89cqfskn\" (UID: \"20409964-6db7-4e50-8369-c031445e2595\") " pod="openshift-ingress/e2e-worker-single-main-85b9fe-openshift-default-58dc79f89cqfskn"
Oct 30 22:33:07.176609 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:07.176376    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/20409964-6db7-4e50-8369-c031445e2595-workload-socket\") pod \"e2e-worker-single-main-85b9fe-openshift-default-58dc79f89cqfskn\" (UID: \"20409964-6db7-4e50-8369-c031445e2595\") " pod="openshift-ingress/e2e-worker-single-main-85b9fe-openshift-default-58dc79f89cqfskn"
Oct 30 22:33:07.176609 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:07.176401    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/20409964-6db7-4e50-8369-c031445e2595-credential-socket\") pod \"e2e-worker-single-main-85b9fe-openshift-default-58dc79f89cqfskn\" (UID: \"20409964-6db7-4e50-8369-c031445e2595\") " pod="openshift-ingress/e2e-worker-single-main-85b9fe-openshift-default-58dc79f89cqfskn"
Oct 30 22:33:07.176723 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:07.176608    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/20409964-6db7-4e50-8369-c031445e2595-istio-data\") pod \"e2e-worker-single-main-85b9fe-openshift-default-58dc79f89cqfskn\" (UID: \"20409964-6db7-4e50-8369-c031445e2595\") " pod="openshift-ingress/e2e-worker-single-main-85b9fe-openshift-default-58dc79f89cqfskn"
Oct 30 22:33:07.176789 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:07.176758    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/20409964-6db7-4e50-8369-c031445e2595-workload-certs\") pod \"e2e-worker-single-main-85b9fe-openshift-default-58dc79f89cqfskn\" (UID: \"20409964-6db7-4e50-8369-c031445e2595\") " pod="openshift-ingress/e2e-worker-single-main-85b9fe-openshift-default-58dc79f89cqfskn"
Oct 30 22:33:07.177136 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:07.177112    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/20409964-6db7-4e50-8369-c031445e2595-istiod-ca-cert\") pod \"e2e-worker-single-main-85b9fe-openshift-default-58dc79f89cqfskn\" (UID: \"20409964-6db7-4e50-8369-c031445e2595\") " pod="openshift-ingress/e2e-worker-single-main-85b9fe-openshift-default-58dc79f89cqfskn"
Oct 30 22:33:07.178828 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:07.178805    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/20409964-6db7-4e50-8369-c031445e2595-istio-envoy\") pod \"e2e-worker-single-main-85b9fe-openshift-default-58dc79f89cqfskn\" (UID: \"20409964-6db7-4e50-8369-c031445e2595\") " pod="openshift-ingress/e2e-worker-single-main-85b9fe-openshift-default-58dc79f89cqfskn"
Oct 30 22:33:07.179066 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:07.179044    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/20409964-6db7-4e50-8369-c031445e2595-istio-podinfo\") pod \"e2e-worker-single-main-85b9fe-openshift-default-58dc79f89cqfskn\" (UID: \"20409964-6db7-4e50-8369-c031445e2595\") " pod="openshift-ingress/e2e-worker-single-main-85b9fe-openshift-default-58dc79f89cqfskn"
Oct 30 22:33:07.233532 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:07.233493    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/20409964-6db7-4e50-8369-c031445e2595-istio-token\") pod \"e2e-worker-single-main-85b9fe-openshift-default-58dc79f89cqfskn\" (UID: \"20409964-6db7-4e50-8369-c031445e2595\") " pod="openshift-ingress/e2e-worker-single-main-85b9fe-openshift-default-58dc79f89cqfskn"
Oct 30 22:33:07.233789 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:07.233763    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wfng5\" (UniqueName: \"kubernetes.io/projected/20409964-6db7-4e50-8369-c031445e2595-kube-api-access-wfng5\") pod \"e2e-worker-single-main-85b9fe-openshift-default-58dc79f89cqfskn\" (UID: \"20409964-6db7-4e50-8369-c031445e2595\") " pod="openshift-ingress/e2e-worker-single-main-85b9fe-openshift-default-58dc79f89cqfskn"
Oct 30 22:33:07.343267 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:07.343182    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-single-main-85b9fe-openshift-default-58dc79f89cqfskn"
Oct 30 22:33:07.789377 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:07.789282    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-single-main-85b9fe-openshift-default-58dc79f89cqfskn"]
Oct 30 22:33:07.791985 ip-10-0-140-18 kubenswrapper[4473]: W1030 22:33:07.791958    4473 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20409964_6db7_4e50_8369_c031445e2595.slice/crio-69b432ac15d2d54d4ddf45722e3e2d150461f43c4eea239c4a363a4926cd1480 WatchSource:0}: Error finding container 69b432ac15d2d54d4ddf45722e3e2d150461f43c4eea239c4a363a4926cd1480: Status 404 returned error can't find the container with id 69b432ac15d2d54d4ddf45722e3e2d150461f43c4eea239c4a363a4926cd1480
Oct 30 22:33:07.794173 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:07.794141    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:33:07.794243 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:07.794209    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:33:07.794243 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:07.794239    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:33:07.882493 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:07.882456    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-85b9fe-openshift-default-58dc79f89cqfskn" event={"ID":"20409964-6db7-4e50-8369-c031445e2595","Type":"ContainerStarted","Data":"4da74d7bdbc510d860e4b5d116ff270435712446c6abe8bcc6fcc95fbc445992"}
Oct 30 22:33:07.882674 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:07.882519    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-85b9fe-openshift-default-58dc79f89cqfskn" event={"ID":"20409964-6db7-4e50-8369-c031445e2595","Type":"ContainerStarted","Data":"69b432ac15d2d54d4ddf45722e3e2d150461f43c4eea239c4a363a4926cd1480"}
Oct 30 22:33:08.005007 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:08.004950    4473 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-single-main-85b9fe-openshift-default-58dc79f89cqfskn" podStartSLOduration=2.004934944 podStartE2EDuration="2.004934944s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:33:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:33:08.003259902 +0000 UTC m=+3033.776628409" watchObservedRunningTime="2026-10-30 22:33:08.004934944 +0000 UTC m=+3033.778303422"
Oct 30 22:33:08.344094 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:08.344054    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-single-main-85b9fe-openshift-default-58dc79f89cqfskn"
Oct 30 22:33:08.355200 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:08.355173    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-single-main-85b9fe-openshift-default-58dc79f89cqfskn"
Oct 30 22:33:08.886956 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:08.886897    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-single-main-85b9fe-openshift-default-58dc79f89cqfskn"
Oct 30 22:33:08.887842 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:08.887822    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-single-main-85b9fe-openshift-default-58dc79f89cqfskn"
Oct 30 22:33:14.516468 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:14.516424    4473 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-85b9fe-7cc9c7dx82"]
Oct 30 22:33:14.520131 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:14.520110    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-85b9fe-7cc9c7dx82"
Oct 30 22:33:14.627242 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:14.624860    4473 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-single-main-85b9fe\""
Oct 30 22:33:14.627242 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:14.624884    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-single-m-bb9a23ba-dockercfg-b4trw\""
Oct 30 22:33:14.628140 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:14.628113    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-85b9fe-7cc9c7dx82"]
Oct 30 22:33:14.652453 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:14.652419    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c6qhk\" (UniqueName: \"kubernetes.io/projected/b2d3ef0a-6dba-482d-a6da-e5405a8bffd1-kube-api-access-c6qhk\") pod \"payload-pre-processing-e2e-worker-single-main-85b9fe-7cc9c7dx82\" (UID: \"b2d3ef0a-6dba-482d-a6da-e5405a8bffd1\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-85b9fe-7cc9c7dx82"
Oct 30 22:33:14.652677 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:14.652513    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b2d3ef0a-6dba-482d-a6da-e5405a8bffd1-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-single-main-85b9fe-7cc9c7dx82\" (UID: \"b2d3ef0a-6dba-482d-a6da-e5405a8bffd1\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-85b9fe-7cc9c7dx82"
Oct 30 22:33:14.661517 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:14.661487    4473 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-85b9fe-868c46bf8kkmmx"]
Oct 30 22:33:14.665066 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:14.665046    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-single-main-85b9fe-868c46bf8kkmmx"
Oct 30 22:33:14.721399 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:14.721361    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-85b9fe-868c46bf8kkmmx"]
Oct 30 22:33:14.753778 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:14.753735    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c6qhk\" (UniqueName: \"kubernetes.io/projected/b2d3ef0a-6dba-482d-a6da-e5405a8bffd1-kube-api-access-c6qhk\") pod \"payload-pre-processing-e2e-worker-single-main-85b9fe-7cc9c7dx82\" (UID: \"b2d3ef0a-6dba-482d-a6da-e5405a8bffd1\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-85b9fe-7cc9c7dx82"
Oct 30 22:33:14.754010 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:14.753795    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/983a880b-b7a6-4a47-b193-b08d5de7e75f-plugins-config-volume\") pod \"payload-processing-e2e-worker-single-main-85b9fe-868c46bf8kkmmx\" (UID: \"983a880b-b7a6-4a47-b193-b08d5de7e75f\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-85b9fe-868c46bf8kkmmx"
Oct 30 22:33:14.754010 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:14.753835    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j5fn4\" (UniqueName: \"kubernetes.io/projected/983a880b-b7a6-4a47-b193-b08d5de7e75f-kube-api-access-j5fn4\") pod \"payload-processing-e2e-worker-single-main-85b9fe-868c46bf8kkmmx\" (UID: \"983a880b-b7a6-4a47-b193-b08d5de7e75f\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-85b9fe-868c46bf8kkmmx"
Oct 30 22:33:14.754010 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:14.753854    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b2d3ef0a-6dba-482d-a6da-e5405a8bffd1-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-single-main-85b9fe-7cc9c7dx82\" (UID: \"b2d3ef0a-6dba-482d-a6da-e5405a8bffd1\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-85b9fe-7cc9c7dx82"
Oct 30 22:33:14.754590 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:14.754569    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b2d3ef0a-6dba-482d-a6da-e5405a8bffd1-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-single-main-85b9fe-7cc9c7dx82\" (UID: \"b2d3ef0a-6dba-482d-a6da-e5405a8bffd1\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-85b9fe-7cc9c7dx82"
Oct 30 22:33:14.811762 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:14.811686    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c6qhk\" (UniqueName: \"kubernetes.io/projected/b2d3ef0a-6dba-482d-a6da-e5405a8bffd1-kube-api-access-c6qhk\") pod \"payload-pre-processing-e2e-worker-single-main-85b9fe-7cc9c7dx82\" (UID: \"b2d3ef0a-6dba-482d-a6da-e5405a8bffd1\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-85b9fe-7cc9c7dx82"
Oct 30 22:33:14.834366 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:14.834336    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-85b9fe-7cc9c7dx82"
Oct 30 22:33:14.855249 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:14.855209    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/983a880b-b7a6-4a47-b193-b08d5de7e75f-plugins-config-volume\") pod \"payload-processing-e2e-worker-single-main-85b9fe-868c46bf8kkmmx\" (UID: \"983a880b-b7a6-4a47-b193-b08d5de7e75f\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-85b9fe-868c46bf8kkmmx"
Oct 30 22:33:14.855442 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:14.855287    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j5fn4\" (UniqueName: \"kubernetes.io/projected/983a880b-b7a6-4a47-b193-b08d5de7e75f-kube-api-access-j5fn4\") pod \"payload-processing-e2e-worker-single-main-85b9fe-868c46bf8kkmmx\" (UID: \"983a880b-b7a6-4a47-b193-b08d5de7e75f\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-85b9fe-868c46bf8kkmmx"
Oct 30 22:33:14.856144 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:14.856108    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/983a880b-b7a6-4a47-b193-b08d5de7e75f-plugins-config-volume\") pod \"payload-processing-e2e-worker-single-main-85b9fe-868c46bf8kkmmx\" (UID: \"983a880b-b7a6-4a47-b193-b08d5de7e75f\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-85b9fe-868c46bf8kkmmx"
Oct 30 22:33:14.915320 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:14.915285    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j5fn4\" (UniqueName: \"kubernetes.io/projected/983a880b-b7a6-4a47-b193-b08d5de7e75f-kube-api-access-j5fn4\") pod \"payload-processing-e2e-worker-single-main-85b9fe-868c46bf8kkmmx\" (UID: \"983a880b-b7a6-4a47-b193-b08d5de7e75f\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-85b9fe-868c46bf8kkmmx"
Oct 30 22:33:14.979462 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:14.979417    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-single-main-85b9fe-868c46bf8kkmmx"
Oct 30 22:33:15.212498 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:15.212470    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-85b9fe-7cc9c7dx82"]
Oct 30 22:33:15.214689 ip-10-0-140-18 kubenswrapper[4473]: W1030 22:33:15.214658    4473 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb2d3ef0a_6dba_482d_a6da_e5405a8bffd1.slice/crio-d9c7af0e45871c3fe59d75e261226f795401b1c68fe86df0f55722257d190e24 WatchSource:0}: Error finding container d9c7af0e45871c3fe59d75e261226f795401b1c68fe86df0f55722257d190e24: Status 404 returned error can't find the container with id d9c7af0e45871c3fe59d75e261226f795401b1c68fe86df0f55722257d190e24
Oct 30 22:33:15.354072 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:15.354043    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-85b9fe-868c46bf8kkmmx"]
Oct 30 22:33:15.355644 ip-10-0-140-18 kubenswrapper[4473]: W1030 22:33:15.355617    4473 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod983a880b_b7a6_4a47_b193_b08d5de7e75f.slice/crio-1049be2ae3a50c418f559e77f91feb8aba9326c128c87d466825f1211719185d WatchSource:0}: Error finding container 1049be2ae3a50c418f559e77f91feb8aba9326c128c87d466825f1211719185d: Status 404 returned error can't find the container with id 1049be2ae3a50c418f559e77f91feb8aba9326c128c87d466825f1211719185d
Oct 30 22:33:15.923891 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:15.923853    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-single-main-85b9fe-868c46bf8kkmmx" event={"ID":"983a880b-b7a6-4a47-b193-b08d5de7e75f","Type":"ContainerStarted","Data":"70f6c70347002924a5426d62a083326c6bdcc65df24b83876448c7e81ec5a882"}
Oct 30 22:33:15.924379 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:15.923949    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-single-main-85b9fe-868c46bf8kkmmx" event={"ID":"983a880b-b7a6-4a47-b193-b08d5de7e75f","Type":"ContainerStarted","Data":"1049be2ae3a50c418f559e77f91feb8aba9326c128c87d466825f1211719185d"}
Oct 30 22:33:15.924379 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:15.923986    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-single-main-85b9fe-868c46bf8kkmmx"
Oct 30 22:33:15.925701 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:15.925677    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-85b9fe-7cc9c7dx82" event={"ID":"b2d3ef0a-6dba-482d-a6da-e5405a8bffd1","Type":"ContainerStarted","Data":"9da8fc0befe30f4b413d50c83081c25b0567c176a1a4d997e03f1492005a11f9"}
Oct 30 22:33:15.925807 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:15.925719    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-85b9fe-7cc9c7dx82" event={"ID":"b2d3ef0a-6dba-482d-a6da-e5405a8bffd1","Type":"ContainerStarted","Data":"d9c7af0e45871c3fe59d75e261226f795401b1c68fe86df0f55722257d190e24"}
Oct 30 22:33:15.925807 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:15.925766    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-85b9fe-7cc9c7dx82"
Oct 30 22:33:16.040748 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:16.040686    4473 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-single-main-85b9fe-868c46bf8kkmmx" podStartSLOduration=2.040670467 podStartE2EDuration="2.040670467s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:33:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:33:16.038225722 +0000 UTC m=+3041.811594210" watchObservedRunningTime="2026-10-30 22:33:16.040670467 +0000 UTC m=+3041.814038953"
Oct 30 22:33:17.214065 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:17.214006    4473 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-85b9fe-7cc9c7dx82" podStartSLOduration=3.2139900949999998 podStartE2EDuration="3.213990095s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:33:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:33:16.155400563 +0000 UTC m=+3041.928769049" watchObservedRunningTime="2026-10-30 22:33:17.213990095 +0000 UTC m=+3042.987358608"
Oct 30 22:33:17.214756 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:17.214737    4473 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-85b9fe-855b5fcd6f-xf6mx"]
Oct 30 22:33:17.218483 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:17.218458    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-85b9fe-855b5fcd6f-xf6mx"
Oct 30 22:33:17.272749 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:17.272710    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-single-main-85b9fe-serving-cert\""
Oct 30 22:33:17.278583 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:17.278544    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/03de7347-3b99-4a45-a1a0-7d2bc7a103c6-metrics-certs\") pod \"maas-api-e2e-worker-single-main-85b9fe-855b5fcd6f-xf6mx\" (UID: \"03de7347-3b99-4a45-a1a0-7d2bc7a103c6\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-85b9fe-855b5fcd6f-xf6mx"
Oct 30 22:33:17.278754 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:17.278694    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/03de7347-3b99-4a45-a1a0-7d2bc7a103c6-openshift-service-ca\") pod \"maas-api-e2e-worker-single-main-85b9fe-855b5fcd6f-xf6mx\" (UID: \"03de7347-3b99-4a45-a1a0-7d2bc7a103c6\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-85b9fe-855b5fcd6f-xf6mx"
Oct 30 22:33:17.278810 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:17.278770    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/03de7347-3b99-4a45-a1a0-7d2bc7a103c6-maas-api-tls\") pod \"maas-api-e2e-worker-single-main-85b9fe-855b5fcd6f-xf6mx\" (UID: \"03de7347-3b99-4a45-a1a0-7d2bc7a103c6\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-85b9fe-855b5fcd6f-xf6mx"
Oct 30 22:33:17.278853 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:17.278816    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lbpjw\" (UniqueName: \"kubernetes.io/projected/03de7347-3b99-4a45-a1a0-7d2bc7a103c6-kube-api-access-lbpjw\") pod \"maas-api-e2e-worker-single-main-85b9fe-855b5fcd6f-xf6mx\" (UID: \"03de7347-3b99-4a45-a1a0-7d2bc7a103c6\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-85b9fe-855b5fcd6f-xf6mx"
Oct 30 22:33:17.326065 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:17.326014    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-85b9fe-855b5fcd6f-xf6mx"]
Oct 30 22:33:17.380284 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:17.380226    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/03de7347-3b99-4a45-a1a0-7d2bc7a103c6-openshift-service-ca\") pod \"maas-api-e2e-worker-single-main-85b9fe-855b5fcd6f-xf6mx\" (UID: \"03de7347-3b99-4a45-a1a0-7d2bc7a103c6\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-85b9fe-855b5fcd6f-xf6mx"
Oct 30 22:33:17.380478 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:17.380346    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/03de7347-3b99-4a45-a1a0-7d2bc7a103c6-maas-api-tls\") pod \"maas-api-e2e-worker-single-main-85b9fe-855b5fcd6f-xf6mx\" (UID: \"03de7347-3b99-4a45-a1a0-7d2bc7a103c6\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-85b9fe-855b5fcd6f-xf6mx"
Oct 30 22:33:17.380478 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:17.380406    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lbpjw\" (UniqueName: \"kubernetes.io/projected/03de7347-3b99-4a45-a1a0-7d2bc7a103c6-kube-api-access-lbpjw\") pod \"maas-api-e2e-worker-single-main-85b9fe-855b5fcd6f-xf6mx\" (UID: \"03de7347-3b99-4a45-a1a0-7d2bc7a103c6\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-85b9fe-855b5fcd6f-xf6mx"
Oct 30 22:33:17.380609 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:33:17.380488    4473 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-single-main-85b9fe-serving-cert: secret "maas-api-e2e-worker-single-main-85b9fe-serving-cert" not found
Oct 30 22:33:17.380609 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:17.380522    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/03de7347-3b99-4a45-a1a0-7d2bc7a103c6-metrics-certs\") pod \"maas-api-e2e-worker-single-main-85b9fe-855b5fcd6f-xf6mx\" (UID: \"03de7347-3b99-4a45-a1a0-7d2bc7a103c6\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-85b9fe-855b5fcd6f-xf6mx"
Oct 30 22:33:17.380609 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:33:17.380563    4473 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/03de7347-3b99-4a45-a1a0-7d2bc7a103c6-maas-api-tls podName:03de7347-3b99-4a45-a1a0-7d2bc7a103c6 nodeName:}" failed. No retries permitted until 2026-10-30 22:33:17.880541724 +0000 UTC m=+3043.653910400 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/03de7347-3b99-4a45-a1a0-7d2bc7a103c6-maas-api-tls") pod "maas-api-e2e-worker-single-main-85b9fe-855b5fcd6f-xf6mx" (UID: "03de7347-3b99-4a45-a1a0-7d2bc7a103c6") : secret "maas-api-e2e-worker-single-main-85b9fe-serving-cert" not found
Oct 30 22:33:17.381107 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:17.381085    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/03de7347-3b99-4a45-a1a0-7d2bc7a103c6-openshift-service-ca\") pod \"maas-api-e2e-worker-single-main-85b9fe-855b5fcd6f-xf6mx\" (UID: \"03de7347-3b99-4a45-a1a0-7d2bc7a103c6\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-85b9fe-855b5fcd6f-xf6mx"
Oct 30 22:33:17.383157 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:17.383134    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/03de7347-3b99-4a45-a1a0-7d2bc7a103c6-metrics-certs\") pod \"maas-api-e2e-worker-single-main-85b9fe-855b5fcd6f-xf6mx\" (UID: \"03de7347-3b99-4a45-a1a0-7d2bc7a103c6\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-85b9fe-855b5fcd6f-xf6mx"
Oct 30 22:33:17.438972 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:17.438935    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lbpjw\" (UniqueName: \"kubernetes.io/projected/03de7347-3b99-4a45-a1a0-7d2bc7a103c6-kube-api-access-lbpjw\") pod \"maas-api-e2e-worker-single-main-85b9fe-855b5fcd6f-xf6mx\" (UID: \"03de7347-3b99-4a45-a1a0-7d2bc7a103c6\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-85b9fe-855b5fcd6f-xf6mx"
Oct 30 22:33:17.886702 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:17.886655    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/03de7347-3b99-4a45-a1a0-7d2bc7a103c6-maas-api-tls\") pod \"maas-api-e2e-worker-single-main-85b9fe-855b5fcd6f-xf6mx\" (UID: \"03de7347-3b99-4a45-a1a0-7d2bc7a103c6\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-85b9fe-855b5fcd6f-xf6mx"
Oct 30 22:33:17.889625 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:17.889594    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/03de7347-3b99-4a45-a1a0-7d2bc7a103c6-maas-api-tls\") pod \"maas-api-e2e-worker-single-main-85b9fe-855b5fcd6f-xf6mx\" (UID: \"03de7347-3b99-4a45-a1a0-7d2bc7a103c6\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-85b9fe-855b5fcd6f-xf6mx"
Oct 30 22:33:18.132219 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:18.132186    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-85b9fe-855b5fcd6f-xf6mx"
Oct 30 22:33:18.509476 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:18.509440    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-85b9fe-855b5fcd6f-xf6mx"]
Oct 30 22:33:18.511073 ip-10-0-140-18 kubenswrapper[4473]: W1030 22:33:18.511042    4473 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod03de7347_3b99_4a45_a1a0_7d2bc7a103c6.slice/crio-fae8431d5783d3f6f202073d57157a654194ea25da5540a923cb1fb7cd46870b WatchSource:0}: Error finding container fae8431d5783d3f6f202073d57157a654194ea25da5540a923cb1fb7cd46870b: Status 404 returned error can't find the container with id fae8431d5783d3f6f202073d57157a654194ea25da5540a923cb1fb7cd46870b
Oct 30 22:33:18.941349 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:18.941313    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-85b9fe-855b5fcd6f-xf6mx" event={"ID":"03de7347-3b99-4a45-a1a0-7d2bc7a103c6","Type":"ContainerStarted","Data":"fae8431d5783d3f6f202073d57157a654194ea25da5540a923cb1fb7cd46870b"}
Oct 30 22:33:19.948138 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:19.948097    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-85b9fe-855b5fcd6f-xf6mx" event={"ID":"03de7347-3b99-4a45-a1a0-7d2bc7a103c6","Type":"ContainerStarted","Data":"a656de0adc8e7d50dc6ea28b3c972f5e02b4739ff60eaea80fe6b3bc346606e3"}
Oct 30 22:33:19.948606 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:19.948197    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-85b9fe-855b5fcd6f-xf6mx"
Oct 30 22:33:20.069968 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:20.069877    4473 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-85b9fe-855b5fcd6f-xf6mx" podStartSLOduration=2.218310422 podStartE2EDuration="3.069861732s" totalImagesPullingTime="851.55131ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:33:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:33:20.06875889 +0000 UTC m=+3045.842127376" watchObservedRunningTime="2026-10-30 22:33:20.069861732 +0000 UTC m=+3045.843230218"
Oct 30 22:33:20.324159 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:33:20.324077    4473 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fncvg" containerName="extractor"
Oct 30 22:33:25.963499 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:25.963463    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-85b9fe-855b5fcd6f-xf6mx"
Oct 30 22:33:26.952784 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:26.952752    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-85b9fe-7cc9c7dx82"
Oct 30 22:33:26.953261 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:26.953239    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-single-main-85b9fe-868c46bf8kkmmx"
Oct 30 22:33:36.030804 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:36.030767    4473 generic.go:356] "Generic (PLEG): container finished" podID="20409964-6db7-4e50-8369-c031445e2595" containerID="4da74d7bdbc510d860e4b5d116ff270435712446c6abe8bcc6fcc95fbc445992" exitCode=0
Oct 30 22:33:36.031318 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:36.030842    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-85b9fe-openshift-default-58dc79f89cqfskn" event={"ID":"20409964-6db7-4e50-8369-c031445e2595","Type":"ContainerDied","Data":"4da74d7bdbc510d860e4b5d116ff270435712446c6abe8bcc6fcc95fbc445992"}
Oct 30 22:33:36.031318 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:36.031246    4473 scope.go:122] "RemoveContainer" containerID="4da74d7bdbc510d860e4b5d116ff270435712446c6abe8bcc6fcc95fbc445992"
Oct 30 22:33:36.032447 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:36.032418    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:33:36.032503 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:36.032486    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:33:36.032540 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:36.032518    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:33:37.057869 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:37.057835    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-85b9fe-openshift-default-58dc79f89cqfskn" event={"ID":"20409964-6db7-4e50-8369-c031445e2595","Type":"ContainerStarted","Data":"34c5c2567121eb6122c5a24a0c97b42ab5ca09ccedfc6b45addc86daac9925ab"}
Oct 30 22:33:37.348481 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:37.348400    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-single-main-85b9fe-openshift-default-58dc79f89cqfskn"
Oct 30 22:33:37.348807 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:37.348785    4473 patch_prober.go:29] interesting pod/e2e-worker-single-main-85b9fe-openshift-default-58dc79f89cqfskn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.155:15021/healthz/ready\": dial tcp 10.133.0.155:15021: connect: connection refused" start-of-body=
Oct 30 22:33:37.348975 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:37.348950    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-85b9fe-openshift-default-58dc79f89cqfskn" podUID="20409964-6db7-4e50-8369-c031445e2595" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.155:15021/healthz/ready\": dial tcp 10.133.0.155:15021: connect: connection refused"
Oct 30 22:33:38.350437 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:38.350399    4473 patch_prober.go:29] interesting pod/e2e-worker-single-main-85b9fe-openshift-default-58dc79f89cqfskn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.155:15021/healthz/ready\": dial tcp 10.133.0.155:15021: connect: connection refused" start-of-body=
Oct 30 22:33:38.351103 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:38.350690    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-85b9fe-openshift-default-58dc79f89cqfskn" podUID="20409964-6db7-4e50-8369-c031445e2595" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.155:15021/healthz/ready\": dial tcp 10.133.0.155:15021: connect: connection refused"
Oct 30 22:33:38.887502 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:38.887462    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-single-main-85b9fe-openshift-default-58dc79f89cqfskn"
Oct 30 22:33:38.904473 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:38.904438    4473 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-single-main-85b9fe/simulator-single-main-85b9fe-kserve-5d5cb9c989-gg5m8"]
Oct 30 22:33:38.906899 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:38.906882    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-single-main-85b9fe/simulator-single-main-85b9fe-kserve-5d5cb9c989-gg5m8"
Oct 30 22:33:39.012221 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:39.012185    4473 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-single-main-85b9fe\"/\"openshift-service-ca.crt\""
Oct 30 22:33:39.012221 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:39.012185    4473 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-single-main-85b9fe\"/\"kube-root-ca.crt\""
Oct 30 22:33:39.012581 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:39.012295    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-single-main-85b9fe\"/\"simulator-single-main-85b9fe-kserve-self-signed-certs\""
Oct 30 22:33:39.012581 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:39.012466    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-single-main-85b9fe\"/\"default-dockercfg-zf5hg\""
Oct 30 22:33:39.014424 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:39.014401    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-single-main-85b9fe/simulator-single-main-85b9fe-kserve-5d5cb9c989-gg5m8"]
Oct 30 22:33:39.056534 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:39.056479    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4382df8f-c16a-4e66-a0e2-bcf0582032fb-tmp-dir\") pod \"simulator-single-main-85b9fe-kserve-5d5cb9c989-gg5m8\" (UID: \"4382df8f-c16a-4e66-a0e2-bcf0582032fb\") " pod="e2e-models-e2e-worker-single-main-85b9fe/simulator-single-main-85b9fe-kserve-5d5cb9c989-gg5m8"
Oct 30 22:33:39.056747 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:39.056636    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4382df8f-c16a-4e66-a0e2-bcf0582032fb-home\") pod \"simulator-single-main-85b9fe-kserve-5d5cb9c989-gg5m8\" (UID: \"4382df8f-c16a-4e66-a0e2-bcf0582032fb\") " pod="e2e-models-e2e-worker-single-main-85b9fe/simulator-single-main-85b9fe-kserve-5d5cb9c989-gg5m8"
Oct 30 22:33:39.056831 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:39.056788    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gmfct\" (UniqueName: \"kubernetes.io/projected/4382df8f-c16a-4e66-a0e2-bcf0582032fb-kube-api-access-gmfct\") pod \"simulator-single-main-85b9fe-kserve-5d5cb9c989-gg5m8\" (UID: \"4382df8f-c16a-4e66-a0e2-bcf0582032fb\") " pod="e2e-models-e2e-worker-single-main-85b9fe/simulator-single-main-85b9fe-kserve-5d5cb9c989-gg5m8"
Oct 30 22:33:39.056831 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:39.056821    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4382df8f-c16a-4e66-a0e2-bcf0582032fb-dshm\") pod \"simulator-single-main-85b9fe-kserve-5d5cb9c989-gg5m8\" (UID: \"4382df8f-c16a-4e66-a0e2-bcf0582032fb\") " pod="e2e-models-e2e-worker-single-main-85b9fe/simulator-single-main-85b9fe-kserve-5d5cb9c989-gg5m8"
Oct 30 22:33:39.056989 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:39.056860    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4382df8f-c16a-4e66-a0e2-bcf0582032fb-model-cache\") pod \"simulator-single-main-85b9fe-kserve-5d5cb9c989-gg5m8\" (UID: \"4382df8f-c16a-4e66-a0e2-bcf0582032fb\") " pod="e2e-models-e2e-worker-single-main-85b9fe/simulator-single-main-85b9fe-kserve-5d5cb9c989-gg5m8"
Oct 30 22:33:39.056989 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:39.056885    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4382df8f-c16a-4e66-a0e2-bcf0582032fb-tls-certs\") pod \"simulator-single-main-85b9fe-kserve-5d5cb9c989-gg5m8\" (UID: \"4382df8f-c16a-4e66-a0e2-bcf0582032fb\") " pod="e2e-models-e2e-worker-single-main-85b9fe/simulator-single-main-85b9fe-kserve-5d5cb9c989-gg5m8"
Oct 30 22:33:39.157981 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:39.157881    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4382df8f-c16a-4e66-a0e2-bcf0582032fb-home\") pod \"simulator-single-main-85b9fe-kserve-5d5cb9c989-gg5m8\" (UID: \"4382df8f-c16a-4e66-a0e2-bcf0582032fb\") " pod="e2e-models-e2e-worker-single-main-85b9fe/simulator-single-main-85b9fe-kserve-5d5cb9c989-gg5m8"
Oct 30 22:33:39.158158 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:39.158020    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gmfct\" (UniqueName: \"kubernetes.io/projected/4382df8f-c16a-4e66-a0e2-bcf0582032fb-kube-api-access-gmfct\") pod \"simulator-single-main-85b9fe-kserve-5d5cb9c989-gg5m8\" (UID: \"4382df8f-c16a-4e66-a0e2-bcf0582032fb\") " pod="e2e-models-e2e-worker-single-main-85b9fe/simulator-single-main-85b9fe-kserve-5d5cb9c989-gg5m8"
Oct 30 22:33:39.158158 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:39.158047    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4382df8f-c16a-4e66-a0e2-bcf0582032fb-dshm\") pod \"simulator-single-main-85b9fe-kserve-5d5cb9c989-gg5m8\" (UID: \"4382df8f-c16a-4e66-a0e2-bcf0582032fb\") " pod="e2e-models-e2e-worker-single-main-85b9fe/simulator-single-main-85b9fe-kserve-5d5cb9c989-gg5m8"
Oct 30 22:33:39.158158 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:39.158089    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4382df8f-c16a-4e66-a0e2-bcf0582032fb-model-cache\") pod \"simulator-single-main-85b9fe-kserve-5d5cb9c989-gg5m8\" (UID: \"4382df8f-c16a-4e66-a0e2-bcf0582032fb\") " pod="e2e-models-e2e-worker-single-main-85b9fe/simulator-single-main-85b9fe-kserve-5d5cb9c989-gg5m8"
Oct 30 22:33:39.158158 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:39.158114    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4382df8f-c16a-4e66-a0e2-bcf0582032fb-tls-certs\") pod \"simulator-single-main-85b9fe-kserve-5d5cb9c989-gg5m8\" (UID: \"4382df8f-c16a-4e66-a0e2-bcf0582032fb\") " pod="e2e-models-e2e-worker-single-main-85b9fe/simulator-single-main-85b9fe-kserve-5d5cb9c989-gg5m8"
Oct 30 22:33:39.158379 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:39.158191    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4382df8f-c16a-4e66-a0e2-bcf0582032fb-tmp-dir\") pod \"simulator-single-main-85b9fe-kserve-5d5cb9c989-gg5m8\" (UID: \"4382df8f-c16a-4e66-a0e2-bcf0582032fb\") " pod="e2e-models-e2e-worker-single-main-85b9fe/simulator-single-main-85b9fe-kserve-5d5cb9c989-gg5m8"
Oct 30 22:33:39.158379 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:39.158365    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4382df8f-c16a-4e66-a0e2-bcf0582032fb-home\") pod \"simulator-single-main-85b9fe-kserve-5d5cb9c989-gg5m8\" (UID: \"4382df8f-c16a-4e66-a0e2-bcf0582032fb\") " pod="e2e-models-e2e-worker-single-main-85b9fe/simulator-single-main-85b9fe-kserve-5d5cb9c989-gg5m8"
Oct 30 22:33:39.158535 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:39.158512    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4382df8f-c16a-4e66-a0e2-bcf0582032fb-model-cache\") pod \"simulator-single-main-85b9fe-kserve-5d5cb9c989-gg5m8\" (UID: \"4382df8f-c16a-4e66-a0e2-bcf0582032fb\") " pod="e2e-models-e2e-worker-single-main-85b9fe/simulator-single-main-85b9fe-kserve-5d5cb9c989-gg5m8"
Oct 30 22:33:39.158648 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:39.158626    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4382df8f-c16a-4e66-a0e2-bcf0582032fb-tmp-dir\") pod \"simulator-single-main-85b9fe-kserve-5d5cb9c989-gg5m8\" (UID: \"4382df8f-c16a-4e66-a0e2-bcf0582032fb\") " pod="e2e-models-e2e-worker-single-main-85b9fe/simulator-single-main-85b9fe-kserve-5d5cb9c989-gg5m8"
Oct 30 22:33:39.160676 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:39.160650    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4382df8f-c16a-4e66-a0e2-bcf0582032fb-dshm\") pod \"simulator-single-main-85b9fe-kserve-5d5cb9c989-gg5m8\" (UID: \"4382df8f-c16a-4e66-a0e2-bcf0582032fb\") " pod="e2e-models-e2e-worker-single-main-85b9fe/simulator-single-main-85b9fe-kserve-5d5cb9c989-gg5m8"
Oct 30 22:33:39.160905 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:39.160886    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4382df8f-c16a-4e66-a0e2-bcf0582032fb-tls-certs\") pod \"simulator-single-main-85b9fe-kserve-5d5cb9c989-gg5m8\" (UID: \"4382df8f-c16a-4e66-a0e2-bcf0582032fb\") " pod="e2e-models-e2e-worker-single-main-85b9fe/simulator-single-main-85b9fe-kserve-5d5cb9c989-gg5m8"
Oct 30 22:33:39.216444 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:39.216405    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gmfct\" (UniqueName: \"kubernetes.io/projected/4382df8f-c16a-4e66-a0e2-bcf0582032fb-kube-api-access-gmfct\") pod \"simulator-single-main-85b9fe-kserve-5d5cb9c989-gg5m8\" (UID: \"4382df8f-c16a-4e66-a0e2-bcf0582032fb\") " pod="e2e-models-e2e-worker-single-main-85b9fe/simulator-single-main-85b9fe-kserve-5d5cb9c989-gg5m8"
Oct 30 22:33:39.244948 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:39.239149    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-single-main-85b9fe/simulator-single-main-85b9fe-kserve-5d5cb9c989-gg5m8"
Oct 30 22:33:39.363840 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:39.363305    4473 patch_prober.go:29] interesting pod/e2e-worker-single-main-85b9fe-openshift-default-58dc79f89cqfskn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.155:15021/healthz/ready\": dial tcp 10.133.0.155:15021: connect: connection refused" start-of-body=
Oct 30 22:33:39.363840 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:39.363381    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-85b9fe-openshift-default-58dc79f89cqfskn" podUID="20409964-6db7-4e50-8369-c031445e2595" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.155:15021/healthz/ready\": dial tcp 10.133.0.155:15021: connect: connection refused"
Oct 30 22:33:39.643945 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:39.637011    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-single-main-85b9fe/simulator-single-main-85b9fe-kserve-5d5cb9c989-gg5m8"]
Oct 30 22:33:39.643945 ip-10-0-140-18 kubenswrapper[4473]: W1030 22:33:39.642475    4473 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4382df8f_c16a_4e66_a0e2_bcf0582032fb.slice/crio-abce2cec6848535f8dcd9b1bafdc077f7982a3702989d0a4ca765e34e9eadbd6 WatchSource:0}: Error finding container abce2cec6848535f8dcd9b1bafdc077f7982a3702989d0a4ca765e34e9eadbd6: Status 404 returned error can't find the container with id abce2cec6848535f8dcd9b1bafdc077f7982a3702989d0a4ca765e34e9eadbd6
Oct 30 22:33:40.078540 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:40.078452    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-single-main-85b9fe/simulator-single-main-85b9fe-kserve-5d5cb9c989-gg5m8" event={"ID":"4382df8f-c16a-4e66-a0e2-bcf0582032fb","Type":"ContainerStarted","Data":"637f77b14fbdd0b0c0e7d3b1689ae355f13051ef369c46a07aedc8ee57b439f0"}
Oct 30 22:33:40.078540 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:40.078492    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-single-main-85b9fe/simulator-single-main-85b9fe-kserve-5d5cb9c989-gg5m8" event={"ID":"4382df8f-c16a-4e66-a0e2-bcf0582032fb","Type":"ContainerStarted","Data":"abce2cec6848535f8dcd9b1bafdc077f7982a3702989d0a4ca765e34e9eadbd6"}
Oct 30 22:33:40.206189 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:40.206129    4473 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-single-main-85b9fe/simulator-single-main-85b9fe-kserve-5d5cb9c989-gg5m8" podStartSLOduration=2.2061149860000002 podStartE2EDuration="2.206114986s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:33:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:33:40.203473061 +0000 UTC m=+3065.976841550" watchObservedRunningTime="2026-10-30 22:33:40.206114986 +0000 UTC m=+3065.979483492"
Oct 30 22:33:40.249999 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:40.247050    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-single-main-85b9fe/simulator-single-main-85b9fe-kserve-5d5cb9c989-gg5m8"
Oct 30 22:33:40.347005 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:40.345329    4473 patch_prober.go:29] interesting pod/e2e-worker-single-main-85b9fe-openshift-default-58dc79f89cqfskn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.155:15021/healthz/ready\": dial tcp 10.133.0.155:15021: connect: connection refused" start-of-body=
Oct 30 22:33:40.347005 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:40.345387    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-85b9fe-openshift-default-58dc79f89cqfskn" podUID="20409964-6db7-4e50-8369-c031445e2595" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.155:15021/healthz/ready\": dial tcp 10.133.0.155:15021: connect: connection refused"
Oct 30 22:33:41.346808 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:41.346350    4473 patch_prober.go:29] interesting pod/e2e-worker-single-main-85b9fe-openshift-default-58dc79f89cqfskn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.155:15021/healthz/ready\": dial tcp 10.133.0.155:15021: connect: connection refused" start-of-body=
Oct 30 22:33:41.346808 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:41.346419    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-85b9fe-openshift-default-58dc79f89cqfskn" podUID="20409964-6db7-4e50-8369-c031445e2595" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.155:15021/healthz/ready\": dial tcp 10.133.0.155:15021: connect: connection refused"
Oct 30 22:33:42.348379 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:42.348345    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-single-main-85b9fe-openshift-default-58dc79f89cqfskn"
Oct 30 22:33:42.355195 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:42.355163    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-single-main-85b9fe-openshift-default-58dc79f89cqfskn"
Oct 30 22:33:49.241960 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:49.241878    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-single-main-85b9fe/simulator-single-main-85b9fe-kserve-5d5cb9c989-gg5m8"
Oct 30 22:33:49.254648 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:49.254617    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-single-main-85b9fe/simulator-single-main-85b9fe-kserve-5d5cb9c989-gg5m8"
Oct 30 22:33:50.138417 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:33:50.138388    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-single-main-85b9fe/simulator-single-main-85b9fe-kserve-5d5cb9c989-gg5m8"
Oct 30 22:33:50.323657 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:33:50.323619    4473 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fncvg" containerName="extractor"
Oct 30 22:34:20.324377 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:34:20.324286    4473 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fncvg" containerName="extractor"
Oct 30 22:34:40.588536 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:34:40.588504    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-single-main-85b9fe/simulator-single-main-85b9fe-kserve-5d5cb9c989-gg5m8"]
Oct 30 22:34:47.689144 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:34:47.689104    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-85b9fe-868c46bf8kkmmx"]
Oct 30 22:34:47.794486 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:34:47.794446    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-85b9fe-7cc9c7dx82"]
Oct 30 22:34:50.323836 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:34:50.323803    4473 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fncvg" containerName="extractor"
Oct 30 22:34:52.689691 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:34:52.689652    4473 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-single-main-85b9fe-868c46bf8kkmmx" podUID="983a880b-b7a6-4a47-b193-b08d5de7e75f" containerName="payload-processing" containerID="cri-o://70f6c70347002924a5426d62a083326c6bdcc65df24b83876448c7e81ec5a882" gracePeriod=25
Oct 30 22:34:52.795726 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:34:52.795665    4473 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-85b9fe-7cc9c7dx82" podUID="b2d3ef0a-6dba-482d-a6da-e5405a8bffd1" containerName="payload-pre-processing" containerID="cri-o://9da8fc0befe30f4b413d50c83081c25b0567c176a1a4d997e03f1492005a11f9" gracePeriod=25
Oct 30 22:34:55.079519 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:34:55.079481    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-85b9fe-855b5fcd6f-xf6mx"]
Oct 30 22:34:55.080010 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:34:55.079701    4473 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-85b9fe-855b5fcd6f-xf6mx" podUID="03de7347-3b99-4a45-a1a0-7d2bc7a103c6" containerName="maas-api" containerID="cri-o://a656de0adc8e7d50dc6ea28b3c972f5e02b4739ff60eaea80fe6b3bc346606e3" gracePeriod=30
Oct 30 22:34:55.609314 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:34:55.609268    4473 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-single-main-85b9fe/simulator-single-main-85b9fe-kserve-5d5cb9c989-gg5m8" podUID="4382df8f-c16a-4e66-a0e2-bcf0582032fb" containerName="main" containerID="cri-o://637f77b14fbdd0b0c0e7d3b1689ae355f13051ef369c46a07aedc8ee57b439f0" gracePeriod=45
Oct 30 22:34:55.872442 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:34:55.872417    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-single-main-85b9fe/simulator-single-main-85b9fe-kserve-5d5cb9c989-gg5m8"
Oct 30 22:34:55.957422 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:34:55.957384    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-85b9fe-855b5fcd6f-xf6mx" podUID="03de7347-3b99-4a45-a1a0-7d2bc7a103c6" containerName="maas-api" probeResult="failure" output="Get \"https://10.133.0.158:8443/health\": dial tcp 10.133.0.158:8443: connect: connection refused"
Oct 30 22:34:55.979120 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:34:55.979085    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4382df8f-c16a-4e66-a0e2-bcf0582032fb-kube-api-access-gmfct\" (UniqueName: \"kubernetes.io/projected/4382df8f-c16a-4e66-a0e2-bcf0582032fb-kube-api-access-gmfct\") pod \"4382df8f-c16a-4e66-a0e2-bcf0582032fb\" (UID: \"4382df8f-c16a-4e66-a0e2-bcf0582032fb\") "
Oct 30 22:34:55.979309 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:34:55.979133    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4382df8f-c16a-4e66-a0e2-bcf0582032fb-dshm\" (UniqueName: \"kubernetes.io/empty-dir/4382df8f-c16a-4e66-a0e2-bcf0582032fb-dshm\") pod \"4382df8f-c16a-4e66-a0e2-bcf0582032fb\" (UID: \"4382df8f-c16a-4e66-a0e2-bcf0582032fb\") "
Oct 30 22:34:55.979309 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:34:55.979213    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4382df8f-c16a-4e66-a0e2-bcf0582032fb-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4382df8f-c16a-4e66-a0e2-bcf0582032fb-model-cache\") pod \"4382df8f-c16a-4e66-a0e2-bcf0582032fb\" (UID: \"4382df8f-c16a-4e66-a0e2-bcf0582032fb\") "
Oct 30 22:34:55.979309 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:34:55.979277    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4382df8f-c16a-4e66-a0e2-bcf0582032fb-tls-certs\" (UniqueName: \"kubernetes.io/secret/4382df8f-c16a-4e66-a0e2-bcf0582032fb-tls-certs\") pod \"4382df8f-c16a-4e66-a0e2-bcf0582032fb\" (UID: \"4382df8f-c16a-4e66-a0e2-bcf0582032fb\") "
Oct 30 22:34:55.979309 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:34:55.979296    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4382df8f-c16a-4e66-a0e2-bcf0582032fb-home\" (UniqueName: \"kubernetes.io/empty-dir/4382df8f-c16a-4e66-a0e2-bcf0582032fb-home\") pod \"4382df8f-c16a-4e66-a0e2-bcf0582032fb\" (UID: \"4382df8f-c16a-4e66-a0e2-bcf0582032fb\") "
Oct 30 22:34:55.979556 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:34:55.979324    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4382df8f-c16a-4e66-a0e2-bcf0582032fb-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4382df8f-c16a-4e66-a0e2-bcf0582032fb-tmp-dir\") pod \"4382df8f-c16a-4e66-a0e2-bcf0582032fb\" (UID: \"4382df8f-c16a-4e66-a0e2-bcf0582032fb\") "
Oct 30 22:34:55.979627 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:34:55.979556    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4382df8f-c16a-4e66-a0e2-bcf0582032fb-model-cache" pod "4382df8f-c16a-4e66-a0e2-bcf0582032fb" (UID: "4382df8f-c16a-4e66-a0e2-bcf0582032fb"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:34:55.979682 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:34:55.979661    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4382df8f-c16a-4e66-a0e2-bcf0582032fb-tmp-dir" pod "4382df8f-c16a-4e66-a0e2-bcf0582032fb" (UID: "4382df8f-c16a-4e66-a0e2-bcf0582032fb"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:34:55.979733 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:34:55.979687    4473 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4382df8f-c16a-4e66-a0e2-bcf0582032fb-model-cache\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:34:55.979733 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:34:55.979713    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4382df8f-c16a-4e66-a0e2-bcf0582032fb-home" pod "4382df8f-c16a-4e66-a0e2-bcf0582032fb" (UID: "4382df8f-c16a-4e66-a0e2-bcf0582032fb"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:34:55.981736 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:34:55.981712    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4382df8f-c16a-4e66-a0e2-bcf0582032fb-dshm" pod "4382df8f-c16a-4e66-a0e2-bcf0582032fb" (UID: "4382df8f-c16a-4e66-a0e2-bcf0582032fb"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:34:55.982129 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:34:55.982109    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4382df8f-c16a-4e66-a0e2-bcf0582032fb-tls-certs" pod "4382df8f-c16a-4e66-a0e2-bcf0582032fb" (UID: "4382df8f-c16a-4e66-a0e2-bcf0582032fb"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:34:55.982196 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:34:55.982126    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4382df8f-c16a-4e66-a0e2-bcf0582032fb-kube-api-access-gmfct" pod "4382df8f-c16a-4e66-a0e2-bcf0582032fb" (UID: "4382df8f-c16a-4e66-a0e2-bcf0582032fb"). InnerVolumeSpecName "kube-api-access-gmfct". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:34:56.080551 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:34:56.080514    4473 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4382df8f-c16a-4e66-a0e2-bcf0582032fb-tls-certs\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:34:56.080551 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:34:56.080549    4473 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4382df8f-c16a-4e66-a0e2-bcf0582032fb-home\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:34:56.081016 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:34:56.080562    4473 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4382df8f-c16a-4e66-a0e2-bcf0582032fb-tmp-dir\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:34:56.081016 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:34:56.080575    4473 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gmfct\" (UniqueName: \"kubernetes.io/projected/4382df8f-c16a-4e66-a0e2-bcf0582032fb-kube-api-access-gmfct\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:34:56.081016 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:34:56.080587    4473 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4382df8f-c16a-4e66-a0e2-bcf0582032fb-dshm\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:34:56.323689 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:34:56.323662    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-85b9fe-855b5fcd6f-xf6mx"
Oct 30 22:34:56.383992 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:34:56.383956    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/03de7347-3b99-4a45-a1a0-7d2bc7a103c6-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/03de7347-3b99-4a45-a1a0-7d2bc7a103c6-openshift-service-ca\") pod \"03de7347-3b99-4a45-a1a0-7d2bc7a103c6\" (UID: \"03de7347-3b99-4a45-a1a0-7d2bc7a103c6\") "
Oct 30 22:34:56.383992 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:34:56.383994    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/03de7347-3b99-4a45-a1a0-7d2bc7a103c6-kube-api-access-lbpjw\" (UniqueName: \"kubernetes.io/projected/03de7347-3b99-4a45-a1a0-7d2bc7a103c6-kube-api-access-lbpjw\") pod \"03de7347-3b99-4a45-a1a0-7d2bc7a103c6\" (UID: \"03de7347-3b99-4a45-a1a0-7d2bc7a103c6\") "
Oct 30 22:34:56.384263 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:34:56.384017    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/03de7347-3b99-4a45-a1a0-7d2bc7a103c6-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/03de7347-3b99-4a45-a1a0-7d2bc7a103c6-maas-api-tls\") pod \"03de7347-3b99-4a45-a1a0-7d2bc7a103c6\" (UID: \"03de7347-3b99-4a45-a1a0-7d2bc7a103c6\") "
Oct 30 22:34:56.384263 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:34:56.384061    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/03de7347-3b99-4a45-a1a0-7d2bc7a103c6-metrics-certs\" (UniqueName: \"kubernetes.io/secret/03de7347-3b99-4a45-a1a0-7d2bc7a103c6-metrics-certs\") pod \"03de7347-3b99-4a45-a1a0-7d2bc7a103c6\" (UID: \"03de7347-3b99-4a45-a1a0-7d2bc7a103c6\") "
Oct 30 22:34:56.384403 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:34:56.384377    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/03de7347-3b99-4a45-a1a0-7d2bc7a103c6-openshift-service-ca" pod "03de7347-3b99-4a45-a1a0-7d2bc7a103c6" (UID: "03de7347-3b99-4a45-a1a0-7d2bc7a103c6"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:34:56.386345 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:34:56.386315    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/03de7347-3b99-4a45-a1a0-7d2bc7a103c6-metrics-certs" pod "03de7347-3b99-4a45-a1a0-7d2bc7a103c6" (UID: "03de7347-3b99-4a45-a1a0-7d2bc7a103c6"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:34:56.386486 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:34:56.386353    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/03de7347-3b99-4a45-a1a0-7d2bc7a103c6-maas-api-tls" pod "03de7347-3b99-4a45-a1a0-7d2bc7a103c6" (UID: "03de7347-3b99-4a45-a1a0-7d2bc7a103c6"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:34:56.386486 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:34:56.386389    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/03de7347-3b99-4a45-a1a0-7d2bc7a103c6-kube-api-access-lbpjw" pod "03de7347-3b99-4a45-a1a0-7d2bc7a103c6" (UID: "03de7347-3b99-4a45-a1a0-7d2bc7a103c6"). InnerVolumeSpecName "kube-api-access-lbpjw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:34:56.458365 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:34:56.458279    4473 generic.go:356] "Generic (PLEG): container finished" podID="03de7347-3b99-4a45-a1a0-7d2bc7a103c6" containerID="a656de0adc8e7d50dc6ea28b3c972f5e02b4739ff60eaea80fe6b3bc346606e3" exitCode=0
Oct 30 22:34:56.458365 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:34:56.458336    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-85b9fe-855b5fcd6f-xf6mx"
Oct 30 22:34:56.458612 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:34:56.458367    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-85b9fe-855b5fcd6f-xf6mx" event={"ID":"03de7347-3b99-4a45-a1a0-7d2bc7a103c6","Type":"ContainerDied","Data":"a656de0adc8e7d50dc6ea28b3c972f5e02b4739ff60eaea80fe6b3bc346606e3"}
Oct 30 22:34:56.458612 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:34:56.458446    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-85b9fe-855b5fcd6f-xf6mx" event={"ID":"03de7347-3b99-4a45-a1a0-7d2bc7a103c6","Type":"ContainerDied","Data":"fae8431d5783d3f6f202073d57157a654194ea25da5540a923cb1fb7cd46870b"}
Oct 30 22:34:56.458612 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:34:56.458475    4473 scope.go:122] "RemoveContainer" containerID="a656de0adc8e7d50dc6ea28b3c972f5e02b4739ff60eaea80fe6b3bc346606e3"
Oct 30 22:34:56.459852 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:34:56.459814    4473 generic.go:356] "Generic (PLEG): container finished" podID="4382df8f-c16a-4e66-a0e2-bcf0582032fb" containerID="637f77b14fbdd0b0c0e7d3b1689ae355f13051ef369c46a07aedc8ee57b439f0" exitCode=0
Oct 30 22:34:56.459945 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:34:56.459877    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-single-main-85b9fe/simulator-single-main-85b9fe-kserve-5d5cb9c989-gg5m8"
Oct 30 22:34:56.459945 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:34:56.459886    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-single-main-85b9fe/simulator-single-main-85b9fe-kserve-5d5cb9c989-gg5m8" event={"ID":"4382df8f-c16a-4e66-a0e2-bcf0582032fb","Type":"ContainerDied","Data":"637f77b14fbdd0b0c0e7d3b1689ae355f13051ef369c46a07aedc8ee57b439f0"}
Oct 30 22:34:56.459945 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:34:56.459909    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-single-main-85b9fe/simulator-single-main-85b9fe-kserve-5d5cb9c989-gg5m8" event={"ID":"4382df8f-c16a-4e66-a0e2-bcf0582032fb","Type":"ContainerDied","Data":"abce2cec6848535f8dcd9b1bafdc077f7982a3702989d0a4ca765e34e9eadbd6"}
Oct 30 22:34:56.473626 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:34:56.473603    4473 scope.go:122] "RemoveContainer" containerID="a656de0adc8e7d50dc6ea28b3c972f5e02b4739ff60eaea80fe6b3bc346606e3"
Oct 30 22:34:56.474007 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:34:56.473982    4473 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a656de0adc8e7d50dc6ea28b3c972f5e02b4739ff60eaea80fe6b3bc346606e3\": container with ID starting with a656de0adc8e7d50dc6ea28b3c972f5e02b4739ff60eaea80fe6b3bc346606e3 not found: ID does not exist" containerID="a656de0adc8e7d50dc6ea28b3c972f5e02b4739ff60eaea80fe6b3bc346606e3"
Oct 30 22:34:56.474123 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:34:56.474015    4473 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a656de0adc8e7d50dc6ea28b3c972f5e02b4739ff60eaea80fe6b3bc346606e3"} err="failed to get container status \"a656de0adc8e7d50dc6ea28b3c972f5e02b4739ff60eaea80fe6b3bc346606e3\": rpc error: code = NotFound desc = could not find container \"a656de0adc8e7d50dc6ea28b3c972f5e02b4739ff60eaea80fe6b3bc346606e3\": container with ID starting with a656de0adc8e7d50dc6ea28b3c972f5e02b4739ff60eaea80fe6b3bc346606e3 not found: ID does not exist"
Oct 30 22:34:56.474123 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:34:56.474035    4473 scope.go:122] "RemoveContainer" containerID="637f77b14fbdd0b0c0e7d3b1689ae355f13051ef369c46a07aedc8ee57b439f0"
Oct 30 22:34:56.482699 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:34:56.482676    4473 scope.go:122] "RemoveContainer" containerID="637f77b14fbdd0b0c0e7d3b1689ae355f13051ef369c46a07aedc8ee57b439f0"
Oct 30 22:34:56.483050 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:34:56.483029    4473 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"637f77b14fbdd0b0c0e7d3b1689ae355f13051ef369c46a07aedc8ee57b439f0\": container with ID starting with 637f77b14fbdd0b0c0e7d3b1689ae355f13051ef369c46a07aedc8ee57b439f0 not found: ID does not exist" containerID="637f77b14fbdd0b0c0e7d3b1689ae355f13051ef369c46a07aedc8ee57b439f0"
Oct 30 22:34:56.483129 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:34:56.483061    4473 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"637f77b14fbdd0b0c0e7d3b1689ae355f13051ef369c46a07aedc8ee57b439f0"} err="failed to get container status \"637f77b14fbdd0b0c0e7d3b1689ae355f13051ef369c46a07aedc8ee57b439f0\": rpc error: code = NotFound desc = could not find container \"637f77b14fbdd0b0c0e7d3b1689ae355f13051ef369c46a07aedc8ee57b439f0\": container with ID starting with 637f77b14fbdd0b0c0e7d3b1689ae355f13051ef369c46a07aedc8ee57b439f0 not found: ID does not exist"
Oct 30 22:34:56.485608 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:34:56.485585    4473 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/03de7347-3b99-4a45-a1a0-7d2bc7a103c6-openshift-service-ca\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:34:56.485704 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:34:56.485609    4473 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lbpjw\" (UniqueName: \"kubernetes.io/projected/03de7347-3b99-4a45-a1a0-7d2bc7a103c6-kube-api-access-lbpjw\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:34:56.485704 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:34:56.485625    4473 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/03de7347-3b99-4a45-a1a0-7d2bc7a103c6-maas-api-tls\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:34:56.485704 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:34:56.485638    4473 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/03de7347-3b99-4a45-a1a0-7d2bc7a103c6-metrics-certs\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:34:56.624302 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:34:56.624261    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-85b9fe-855b5fcd6f-xf6mx"]
Oct 30 22:34:56.628504 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:34:56.628471    4473 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-85b9fe-855b5fcd6f-xf6mx"]
Oct 30 22:34:56.737526 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:34:56.737442    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-single-main-85b9fe/simulator-single-main-85b9fe-kserve-5d5cb9c989-gg5m8"]
Oct 30 22:34:56.744679 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:34:56.744651    4473 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-single-main-85b9fe/simulator-single-main-85b9fe-kserve-5d5cb9c989-gg5m8"]
Oct 30 22:34:56.932188 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:34:56.932145    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-85b9fe-7cc9c7dx82" podUID="b2d3ef0a-6dba-482d-a6da-e5405a8bffd1" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:34:56.932188 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:34:56.932180    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-single-main-85b9fe-868c46bf8kkmmx" podUID="983a880b-b7a6-4a47-b193-b08d5de7e75f" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:34:58.355601 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:34:58.355561    4473 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="03de7347-3b99-4a45-a1a0-7d2bc7a103c6" path="/var/lib/kubelet/pods/03de7347-3b99-4a45-a1a0-7d2bc7a103c6/volumes"
Oct 30 22:34:58.356033 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:34:58.356020    4473 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4382df8f-c16a-4e66-a0e2-bcf0582032fb" path="/var/lib/kubelet/pods/4382df8f-c16a-4e66-a0e2-bcf0582032fb/volumes"
Oct 30 22:35:04.059836 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:35:04.059802    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-single-main-85b9fe-openshift-default-58dc79f89cqfskn"]
Oct 30 22:35:04.060294 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:35:04.060029    4473 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-single-main-85b9fe-openshift-default-58dc79f89cqfskn" podUID="20409964-6db7-4e50-8369-c031445e2595" containerName="istio-proxy" containerID="cri-o://34c5c2567121eb6122c5a24a0c97b42ab5ca09ccedfc6b45addc86daac9925ab" gracePeriod=30
Oct 30 22:35:06.932407 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:35:06.932355    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-85b9fe-7cc9c7dx82" podUID="b2d3ef0a-6dba-482d-a6da-e5405a8bffd1" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:35:06.932805 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:35:06.932421    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-single-main-85b9fe-868c46bf8kkmmx" podUID="983a880b-b7a6-4a47-b193-b08d5de7e75f" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:35:09.311830 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:35:09.311801    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-single-main-85b9fe-openshift-default-58dc79f89cqfskn"
Oct 30 22:35:09.424315 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:35:09.424271    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/20409964-6db7-4e50-8369-c031445e2595-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/20409964-6db7-4e50-8369-c031445e2595-workload-socket\") pod \"20409964-6db7-4e50-8369-c031445e2595\" (UID: \"20409964-6db7-4e50-8369-c031445e2595\") "
Oct 30 22:35:09.424315 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:35:09.424313    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/20409964-6db7-4e50-8369-c031445e2595-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/20409964-6db7-4e50-8369-c031445e2595-istio-podinfo\") pod \"20409964-6db7-4e50-8369-c031445e2595\" (UID: \"20409964-6db7-4e50-8369-c031445e2595\") "
Oct 30 22:35:09.424608 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:35:09.424363    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/20409964-6db7-4e50-8369-c031445e2595-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/20409964-6db7-4e50-8369-c031445e2595-istiod-ca-cert\") pod \"20409964-6db7-4e50-8369-c031445e2595\" (UID: \"20409964-6db7-4e50-8369-c031445e2595\") "
Oct 30 22:35:09.424608 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:35:09.424434    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/20409964-6db7-4e50-8369-c031445e2595-istio-token\" (UniqueName: \"kubernetes.io/projected/20409964-6db7-4e50-8369-c031445e2595-istio-token\") pod \"20409964-6db7-4e50-8369-c031445e2595\" (UID: \"20409964-6db7-4e50-8369-c031445e2595\") "
Oct 30 22:35:09.424608 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:35:09.424531    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/20409964-6db7-4e50-8369-c031445e2595-kube-api-access-wfng5\" (UniqueName: \"kubernetes.io/projected/20409964-6db7-4e50-8369-c031445e2595-kube-api-access-wfng5\") pod \"20409964-6db7-4e50-8369-c031445e2595\" (UID: \"20409964-6db7-4e50-8369-c031445e2595\") "
Oct 30 22:35:09.424719 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:35:09.424562    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/20409964-6db7-4e50-8369-c031445e2595-workload-socket" pod "20409964-6db7-4e50-8369-c031445e2595" (UID: "20409964-6db7-4e50-8369-c031445e2595"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:35:09.424719 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:35:09.424656    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/20409964-6db7-4e50-8369-c031445e2595-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/20409964-6db7-4e50-8369-c031445e2595-istio-envoy\") pod \"20409964-6db7-4e50-8369-c031445e2595\" (UID: \"20409964-6db7-4e50-8369-c031445e2595\") "
Oct 30 22:35:09.424719 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:35:09.424702    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/20409964-6db7-4e50-8369-c031445e2595-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/20409964-6db7-4e50-8369-c031445e2595-istio-data\") pod \"20409964-6db7-4e50-8369-c031445e2595\" (UID: \"20409964-6db7-4e50-8369-c031445e2595\") "
Oct 30 22:35:09.424841 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:35:09.424735    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/20409964-6db7-4e50-8369-c031445e2595-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/20409964-6db7-4e50-8369-c031445e2595-credential-socket\") pod \"20409964-6db7-4e50-8369-c031445e2595\" (UID: \"20409964-6db7-4e50-8369-c031445e2595\") "
Oct 30 22:35:09.424841 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:35:09.424784    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/20409964-6db7-4e50-8369-c031445e2595-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/20409964-6db7-4e50-8369-c031445e2595-workload-certs\") pod \"20409964-6db7-4e50-8369-c031445e2595\" (UID: \"20409964-6db7-4e50-8369-c031445e2595\") "
Oct 30 22:35:09.424971 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:35:09.424893    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/20409964-6db7-4e50-8369-c031445e2595-istiod-ca-cert" pod "20409964-6db7-4e50-8369-c031445e2595" (UID: "20409964-6db7-4e50-8369-c031445e2595"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:35:09.425040 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:35:09.424997    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/20409964-6db7-4e50-8369-c031445e2595-workload-certs" pod "20409964-6db7-4e50-8369-c031445e2595" (UID: "20409964-6db7-4e50-8369-c031445e2595"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:35:09.425216 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:35:09.425193    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/20409964-6db7-4e50-8369-c031445e2595-istio-data" pod "20409964-6db7-4e50-8369-c031445e2595" (UID: "20409964-6db7-4e50-8369-c031445e2595"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:35:09.425464 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:35:09.425432    4473 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/20409964-6db7-4e50-8369-c031445e2595-istio-data\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:35:09.425817 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:35:09.425792    4473 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/20409964-6db7-4e50-8369-c031445e2595-workload-certs\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:35:09.425957 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:35:09.425819    4473 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/20409964-6db7-4e50-8369-c031445e2595-workload-socket\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:35:09.425957 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:35:09.425835    4473 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/20409964-6db7-4e50-8369-c031445e2595-istiod-ca-cert\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:35:09.425957 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:35:09.425567    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/20409964-6db7-4e50-8369-c031445e2595-credential-socket" pod "20409964-6db7-4e50-8369-c031445e2595" (UID: "20409964-6db7-4e50-8369-c031445e2595"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:35:09.427227 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:35:09.427203    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/20409964-6db7-4e50-8369-c031445e2595-kube-api-access-wfng5" pod "20409964-6db7-4e50-8369-c031445e2595" (UID: "20409964-6db7-4e50-8369-c031445e2595"). InnerVolumeSpecName "kube-api-access-wfng5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:35:09.427387 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:35:09.427365    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/20409964-6db7-4e50-8369-c031445e2595-istio-envoy" pod "20409964-6db7-4e50-8369-c031445e2595" (UID: "20409964-6db7-4e50-8369-c031445e2595"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:35:09.427491 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:35:09.427471    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/20409964-6db7-4e50-8369-c031445e2595-istio-podinfo" pod "20409964-6db7-4e50-8369-c031445e2595" (UID: "20409964-6db7-4e50-8369-c031445e2595"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:35:09.427552 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:35:09.427496    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/20409964-6db7-4e50-8369-c031445e2595-istio-token" pod "20409964-6db7-4e50-8369-c031445e2595" (UID: "20409964-6db7-4e50-8369-c031445e2595"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:35:09.526181 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:35:09.526149    4473 generic.go:356] "Generic (PLEG): container finished" podID="20409964-6db7-4e50-8369-c031445e2595" containerID="34c5c2567121eb6122c5a24a0c97b42ab5ca09ccedfc6b45addc86daac9925ab" exitCode=0
Oct 30 22:35:09.526375 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:35:09.526218    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-single-main-85b9fe-openshift-default-58dc79f89cqfskn"
Oct 30 22:35:09.526375 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:35:09.526241    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-85b9fe-openshift-default-58dc79f89cqfskn" event={"ID":"20409964-6db7-4e50-8369-c031445e2595","Type":"ContainerDied","Data":"34c5c2567121eb6122c5a24a0c97b42ab5ca09ccedfc6b45addc86daac9925ab"}
Oct 30 22:35:09.526375 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:35:09.526348    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-85b9fe-openshift-default-58dc79f89cqfskn" event={"ID":"20409964-6db7-4e50-8369-c031445e2595","Type":"ContainerDied","Data":"69b432ac15d2d54d4ddf45722e3e2d150461f43c4eea239c4a363a4926cd1480"}
Oct 30 22:35:09.526563 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:35:09.526380    4473 scope.go:122] "RemoveContainer" containerID="34c5c2567121eb6122c5a24a0c97b42ab5ca09ccedfc6b45addc86daac9925ab"
Oct 30 22:35:09.526655 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:35:09.526607    4473 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/20409964-6db7-4e50-8369-c031445e2595-istio-envoy\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:35:09.526655 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:35:09.526638    4473 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/20409964-6db7-4e50-8369-c031445e2595-credential-socket\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:35:09.526655 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:35:09.526655    4473 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/20409964-6db7-4e50-8369-c031445e2595-istio-podinfo\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:35:09.526909 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:35:09.526670    4473 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/20409964-6db7-4e50-8369-c031445e2595-istio-token\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:35:09.526909 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:35:09.526684    4473 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wfng5\" (UniqueName: \"kubernetes.io/projected/20409964-6db7-4e50-8369-c031445e2595-kube-api-access-wfng5\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:35:09.536392 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:35:09.536371    4473 scope.go:122] "RemoveContainer" containerID="4da74d7bdbc510d860e4b5d116ff270435712446c6abe8bcc6fcc95fbc445992"
Oct 30 22:35:09.545395 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:35:09.545374    4473 scope.go:122] "RemoveContainer" containerID="34c5c2567121eb6122c5a24a0c97b42ab5ca09ccedfc6b45addc86daac9925ab"
Oct 30 22:35:09.545717 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:35:09.545697    4473 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"34c5c2567121eb6122c5a24a0c97b42ab5ca09ccedfc6b45addc86daac9925ab\": container with ID starting with 34c5c2567121eb6122c5a24a0c97b42ab5ca09ccedfc6b45addc86daac9925ab not found: ID does not exist" containerID="34c5c2567121eb6122c5a24a0c97b42ab5ca09ccedfc6b45addc86daac9925ab"
Oct 30 22:35:09.545779 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:35:09.545727    4473 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"34c5c2567121eb6122c5a24a0c97b42ab5ca09ccedfc6b45addc86daac9925ab"} err="failed to get container status \"34c5c2567121eb6122c5a24a0c97b42ab5ca09ccedfc6b45addc86daac9925ab\": rpc error: code = NotFound desc = could not find container \"34c5c2567121eb6122c5a24a0c97b42ab5ca09ccedfc6b45addc86daac9925ab\": container with ID starting with 34c5c2567121eb6122c5a24a0c97b42ab5ca09ccedfc6b45addc86daac9925ab not found: ID does not exist"
Oct 30 22:35:09.545779 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:35:09.545748    4473 scope.go:122] "RemoveContainer" containerID="4da74d7bdbc510d860e4b5d116ff270435712446c6abe8bcc6fcc95fbc445992"
Oct 30 22:35:09.546030 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:35:09.546013    4473 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4da74d7bdbc510d860e4b5d116ff270435712446c6abe8bcc6fcc95fbc445992\": container with ID starting with 4da74d7bdbc510d860e4b5d116ff270435712446c6abe8bcc6fcc95fbc445992 not found: ID does not exist" containerID="4da74d7bdbc510d860e4b5d116ff270435712446c6abe8bcc6fcc95fbc445992"
Oct 30 22:35:09.546083 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:35:09.546037    4473 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4da74d7bdbc510d860e4b5d116ff270435712446c6abe8bcc6fcc95fbc445992"} err="failed to get container status \"4da74d7bdbc510d860e4b5d116ff270435712446c6abe8bcc6fcc95fbc445992\": rpc error: code = NotFound desc = could not find container \"4da74d7bdbc510d860e4b5d116ff270435712446c6abe8bcc6fcc95fbc445992\": container with ID starting with 4da74d7bdbc510d860e4b5d116ff270435712446c6abe8bcc6fcc95fbc445992 not found: ID does not exist"
Oct 30 22:35:09.662992 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:35:09.662947    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-single-main-85b9fe-openshift-default-58dc79f89cqfskn"]
Oct 30 22:35:09.668947 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:35:09.668901    4473 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-single-main-85b9fe-openshift-default-58dc79f89cqfskn"]
Oct 30 22:35:10.356412 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:35:10.356382    4473 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="20409964-6db7-4e50-8369-c031445e2595" path="/var/lib/kubelet/pods/20409964-6db7-4e50-8369-c031445e2595/volumes"
Oct 30 22:35:12.949398 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:35:12.949329    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-single-main-85b9fe-868c46bf8kkmmx"
Oct 30 22:35:13.045083 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:35:13.045057    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-85b9fe-7cc9c7dx82"
Oct 30 22:35:13.062156 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:35:13.062114    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/983a880b-b7a6-4a47-b193-b08d5de7e75f-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/983a880b-b7a6-4a47-b193-b08d5de7e75f-plugins-config-volume\") pod \"983a880b-b7a6-4a47-b193-b08d5de7e75f\" (UID: \"983a880b-b7a6-4a47-b193-b08d5de7e75f\") "
Oct 30 22:35:13.062356 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:35:13.062238    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/983a880b-b7a6-4a47-b193-b08d5de7e75f-kube-api-access-j5fn4\" (UniqueName: \"kubernetes.io/projected/983a880b-b7a6-4a47-b193-b08d5de7e75f-kube-api-access-j5fn4\") pod \"983a880b-b7a6-4a47-b193-b08d5de7e75f\" (UID: \"983a880b-b7a6-4a47-b193-b08d5de7e75f\") "
Oct 30 22:35:13.062565 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:35:13.062540    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/983a880b-b7a6-4a47-b193-b08d5de7e75f-plugins-config-volume" pod "983a880b-b7a6-4a47-b193-b08d5de7e75f" (UID: "983a880b-b7a6-4a47-b193-b08d5de7e75f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:35:13.064436 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:35:13.064415    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/983a880b-b7a6-4a47-b193-b08d5de7e75f-kube-api-access-j5fn4" pod "983a880b-b7a6-4a47-b193-b08d5de7e75f" (UID: "983a880b-b7a6-4a47-b193-b08d5de7e75f"). InnerVolumeSpecName "kube-api-access-j5fn4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:35:13.163547 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:35:13.163503    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b2d3ef0a-6dba-482d-a6da-e5405a8bffd1-kube-api-access-c6qhk\" (UniqueName: \"kubernetes.io/projected/b2d3ef0a-6dba-482d-a6da-e5405a8bffd1-kube-api-access-c6qhk\") pod \"b2d3ef0a-6dba-482d-a6da-e5405a8bffd1\" (UID: \"b2d3ef0a-6dba-482d-a6da-e5405a8bffd1\") "
Oct 30 22:35:13.163753 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:35:13.163584    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b2d3ef0a-6dba-482d-a6da-e5405a8bffd1-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b2d3ef0a-6dba-482d-a6da-e5405a8bffd1-plugins-config-volume\") pod \"b2d3ef0a-6dba-482d-a6da-e5405a8bffd1\" (UID: \"b2d3ef0a-6dba-482d-a6da-e5405a8bffd1\") "
Oct 30 22:35:13.163873 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:35:13.163859    4473 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/983a880b-b7a6-4a47-b193-b08d5de7e75f-plugins-config-volume\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:35:13.163967 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:35:13.163875    4473 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j5fn4\" (UniqueName: \"kubernetes.io/projected/983a880b-b7a6-4a47-b193-b08d5de7e75f-kube-api-access-j5fn4\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:35:13.164033 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:35:13.163972    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b2d3ef0a-6dba-482d-a6da-e5405a8bffd1-plugins-config-volume" pod "b2d3ef0a-6dba-482d-a6da-e5405a8bffd1" (UID: "b2d3ef0a-6dba-482d-a6da-e5405a8bffd1"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:35:13.165894 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:35:13.165870    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b2d3ef0a-6dba-482d-a6da-e5405a8bffd1-kube-api-access-c6qhk" pod "b2d3ef0a-6dba-482d-a6da-e5405a8bffd1" (UID: "b2d3ef0a-6dba-482d-a6da-e5405a8bffd1"). InnerVolumeSpecName "kube-api-access-c6qhk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:35:13.264640 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:35:13.264553    4473 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b2d3ef0a-6dba-482d-a6da-e5405a8bffd1-plugins-config-volume\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:35:13.264640 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:35:13.264584    4473 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c6qhk\" (UniqueName: \"kubernetes.io/projected/b2d3ef0a-6dba-482d-a6da-e5405a8bffd1-kube-api-access-c6qhk\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:35:13.548310 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:35:13.548207    4473 generic.go:356] "Generic (PLEG): container finished" podID="983a880b-b7a6-4a47-b193-b08d5de7e75f" containerID="70f6c70347002924a5426d62a083326c6bdcc65df24b83876448c7e81ec5a882" exitCode=0
Oct 30 22:35:13.548310 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:35:13.548259    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-single-main-85b9fe-868c46bf8kkmmx"
Oct 30 22:35:13.548558 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:35:13.548247    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-single-main-85b9fe-868c46bf8kkmmx" event={"ID":"983a880b-b7a6-4a47-b193-b08d5de7e75f","Type":"ContainerDied","Data":"70f6c70347002924a5426d62a083326c6bdcc65df24b83876448c7e81ec5a882"}
Oct 30 22:35:13.548558 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:35:13.548408    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-single-main-85b9fe-868c46bf8kkmmx" event={"ID":"983a880b-b7a6-4a47-b193-b08d5de7e75f","Type":"ContainerDied","Data":"1049be2ae3a50c418f559e77f91feb8aba9326c128c87d466825f1211719185d"}
Oct 30 22:35:13.548558 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:35:13.548436    4473 scope.go:122] "RemoveContainer" containerID="70f6c70347002924a5426d62a083326c6bdcc65df24b83876448c7e81ec5a882"
Oct 30 22:35:13.549684 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:35:13.549667    4473 generic.go:356] "Generic (PLEG): container finished" podID="b2d3ef0a-6dba-482d-a6da-e5405a8bffd1" containerID="9da8fc0befe30f4b413d50c83081c25b0567c176a1a4d997e03f1492005a11f9" exitCode=0
Oct 30 22:35:13.549761 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:35:13.549746    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-85b9fe-7cc9c7dx82" event={"ID":"b2d3ef0a-6dba-482d-a6da-e5405a8bffd1","Type":"ContainerDied","Data":"9da8fc0befe30f4b413d50c83081c25b0567c176a1a4d997e03f1492005a11f9"}
Oct 30 22:35:13.549832 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:35:13.549778    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-85b9fe-7cc9c7dx82"
Oct 30 22:35:13.549832 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:35:13.549786    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-85b9fe-7cc9c7dx82" event={"ID":"b2d3ef0a-6dba-482d-a6da-e5405a8bffd1","Type":"ContainerDied","Data":"d9c7af0e45871c3fe59d75e261226f795401b1c68fe86df0f55722257d190e24"}
Oct 30 22:35:13.559039 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:35:13.559000    4473 scope.go:122] "RemoveContainer" containerID="70f6c70347002924a5426d62a083326c6bdcc65df24b83876448c7e81ec5a882"
Oct 30 22:35:13.559367 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:35:13.559340    4473 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"70f6c70347002924a5426d62a083326c6bdcc65df24b83876448c7e81ec5a882\": container with ID starting with 70f6c70347002924a5426d62a083326c6bdcc65df24b83876448c7e81ec5a882 not found: ID does not exist" containerID="70f6c70347002924a5426d62a083326c6bdcc65df24b83876448c7e81ec5a882"
Oct 30 22:35:13.559467 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:35:13.559380    4473 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"70f6c70347002924a5426d62a083326c6bdcc65df24b83876448c7e81ec5a882"} err="failed to get container status \"70f6c70347002924a5426d62a083326c6bdcc65df24b83876448c7e81ec5a882\": rpc error: code = NotFound desc = could not find container \"70f6c70347002924a5426d62a083326c6bdcc65df24b83876448c7e81ec5a882\": container with ID starting with 70f6c70347002924a5426d62a083326c6bdcc65df24b83876448c7e81ec5a882 not found: ID does not exist"
Oct 30 22:35:13.559467 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:35:13.559403    4473 scope.go:122] "RemoveContainer" containerID="9da8fc0befe30f4b413d50c83081c25b0567c176a1a4d997e03f1492005a11f9"
Oct 30 22:35:13.568244 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:35:13.568224    4473 scope.go:122] "RemoveContainer" containerID="9da8fc0befe30f4b413d50c83081c25b0567c176a1a4d997e03f1492005a11f9"
Oct 30 22:35:13.568561 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:35:13.568542    4473 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9da8fc0befe30f4b413d50c83081c25b0567c176a1a4d997e03f1492005a11f9\": container with ID starting with 9da8fc0befe30f4b413d50c83081c25b0567c176a1a4d997e03f1492005a11f9 not found: ID does not exist" containerID="9da8fc0befe30f4b413d50c83081c25b0567c176a1a4d997e03f1492005a11f9"
Oct 30 22:35:13.568610 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:35:13.568572    4473 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9da8fc0befe30f4b413d50c83081c25b0567c176a1a4d997e03f1492005a11f9"} err="failed to get container status \"9da8fc0befe30f4b413d50c83081c25b0567c176a1a4d997e03f1492005a11f9\": rpc error: code = NotFound desc = could not find container \"9da8fc0befe30f4b413d50c83081c25b0567c176a1a4d997e03f1492005a11f9\": container with ID starting with 9da8fc0befe30f4b413d50c83081c25b0567c176a1a4d997e03f1492005a11f9 not found: ID does not exist"
Oct 30 22:35:13.718508 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:35:13.718472    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-85b9fe-7cc9c7dx82"]
Oct 30 22:35:13.768746 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:35:13.768713    4473 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-85b9fe-7cc9c7dx82"]
Oct 30 22:35:13.835680 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:35:13.835592    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-85b9fe-868c46bf8kkmmx"]
Oct 30 22:35:13.841336 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:35:13.841307    4473 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-85b9fe-868c46bf8kkmmx"]
Oct 30 22:35:13.880396 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:35:13.880361    4473 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-23ea8c7f-kserve-6c8b9765f7-s6g4p"]
Oct 30 22:35:13.880782 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:35:13.880768    4473 cpu_manager.go:414] "containerMap: removing container" podUID="03de7347-3b99-4a45-a1a0-7d2bc7a103c6" containerName="maas-api"
Oct 30 22:35:13.880836 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:35:13.880783    4473 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="03de7347-3b99-4a45-a1a0-7d2bc7a103c6" containerName="maas-api"
Oct 30 22:35:13.880836 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:35:13.880795    4473 cpu_manager.go:414] "containerMap: removing container" podUID="983a880b-b7a6-4a47-b193-b08d5de7e75f" containerName="payload-processing"
Oct 30 22:35:13.880836 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:35:13.880800    4473 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="983a880b-b7a6-4a47-b193-b08d5de7e75f" containerName="payload-processing"
Oct 30 22:35:13.880836 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:35:13.880815    4473 cpu_manager.go:414] "containerMap: removing container" podUID="20409964-6db7-4e50-8369-c031445e2595" containerName="istio-proxy"
Oct 30 22:35:13.880836 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:35:13.880820    4473 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="20409964-6db7-4e50-8369-c031445e2595" containerName="istio-proxy"
Oct 30 22:35:13.880836 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:35:13.880831    4473 cpu_manager.go:414] "containerMap: removing container" podUID="b2d3ef0a-6dba-482d-a6da-e5405a8bffd1" containerName="payload-pre-processing"
Oct 30 22:35:13.880836 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:35:13.880835    4473 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b2d3ef0a-6dba-482d-a6da-e5405a8bffd1" containerName="payload-pre-processing"
Oct 30 22:35:13.881119 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:35:13.880852    4473 cpu_manager.go:414] "containerMap: removing container" podUID="4382df8f-c16a-4e66-a0e2-bcf0582032fb" containerName="main"
Oct 30 22:35:13.881119 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:35:13.880857    4473 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4382df8f-c16a-4e66-a0e2-bcf0582032fb" containerName="main"
Oct 30 22:35:13.881119 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:35:13.880912    4473 memory_manager.go:365] "RemoveStaleState removing state" podUID="b2d3ef0a-6dba-482d-a6da-e5405a8bffd1" containerName="payload-pre-processing"
Oct 30 22:35:13.881119 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:35:13.880938    4473 memory_manager.go:365] "RemoveStaleState removing state" podUID="4382df8f-c16a-4e66-a0e2-bcf0582032fb" containerName="main"
Oct 30 22:35:13.881119 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:35:13.880944    4473 memory_manager.go:365] "RemoveStaleState removing state" podUID="20409964-6db7-4e50-8369-c031445e2595" containerName="istio-proxy"
Oct 30 22:35:13.881119 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:35:13.880952    4473 memory_manager.go:365] "RemoveStaleState removing state" podUID="03de7347-3b99-4a45-a1a0-7d2bc7a103c6" containerName="maas-api"
Oct 30 22:35:13.881119 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:35:13.880957    4473 memory_manager.go:365] "RemoveStaleState removing state" podUID="983a880b-b7a6-4a47-b193-b08d5de7e75f" containerName="payload-processing"
Oct 30 22:35:13.881119 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:35:13.880965    4473 memory_manager.go:365] "RemoveStaleState removing state" podUID="20409964-6db7-4e50-8369-c031445e2595" containerName="istio-proxy"
Oct 30 22:35:13.885549 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:35:13.885525    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-23ea8c7f-kserve-6c8b9765f7-s6g4p"
Oct 30 22:35:13.941452 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:35:13.941415    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-23ea8c7f-kserve-self-signed-certs\""
Oct 30 22:35:13.944933 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:35:13.944883    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-23ea8c7f-kserve-6c8b9765f7-s6g4p"]
Oct 30 22:35:14.072114 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:35:14.072075    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b6fe05b3-4371-465c-8935-5b11c0f3a5f6-dshm\") pod \"e2e-same-id-a-23ea8c7f-kserve-6c8b9765f7-s6g4p\" (UID: \"b6fe05b3-4371-465c-8935-5b11c0f3a5f6\") " pod="llm/e2e-same-id-a-23ea8c7f-kserve-6c8b9765f7-s6g4p"
Oct 30 22:35:14.072538 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:35:14.072147    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b6fe05b3-4371-465c-8935-5b11c0f3a5f6-model-cache\") pod \"e2e-same-id-a-23ea8c7f-kserve-6c8b9765f7-s6g4p\" (UID: \"b6fe05b3-4371-465c-8935-5b11c0f3a5f6\") " pod="llm/e2e-same-id-a-23ea8c7f-kserve-6c8b9765f7-s6g4p"
Oct 30 22:35:14.072538 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:35:14.072286    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b6fe05b3-4371-465c-8935-5b11c0f3a5f6-tmp-dir\") pod \"e2e-same-id-a-23ea8c7f-kserve-6c8b9765f7-s6g4p\" (UID: \"b6fe05b3-4371-465c-8935-5b11c0f3a5f6\") " pod="llm/e2e-same-id-a-23ea8c7f-kserve-6c8b9765f7-s6g4p"
Oct 30 22:35:14.072538 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:35:14.072323    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b6fe05b3-4371-465c-8935-5b11c0f3a5f6-home\") pod \"e2e-same-id-a-23ea8c7f-kserve-6c8b9765f7-s6g4p\" (UID: \"b6fe05b3-4371-465c-8935-5b11c0f3a5f6\") " pod="llm/e2e-same-id-a-23ea8c7f-kserve-6c8b9765f7-s6g4p"
Oct 30 22:35:14.072538 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:35:14.072342    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b6fe05b3-4371-465c-8935-5b11c0f3a5f6-tls-certs\") pod \"e2e-same-id-a-23ea8c7f-kserve-6c8b9765f7-s6g4p\" (UID: \"b6fe05b3-4371-465c-8935-5b11c0f3a5f6\") " pod="llm/e2e-same-id-a-23ea8c7f-kserve-6c8b9765f7-s6g4p"
Oct 30 22:35:14.072538 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:35:14.072391    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sq2m8\" (UniqueName: \"kubernetes.io/projected/b6fe05b3-4371-465c-8935-5b11c0f3a5f6-kube-api-access-sq2m8\") pod \"e2e-same-id-a-23ea8c7f-kserve-6c8b9765f7-s6g4p\" (UID: \"b6fe05b3-4371-465c-8935-5b11c0f3a5f6\") " pod="llm/e2e-same-id-a-23ea8c7f-kserve-6c8b9765f7-s6g4p"
Oct 30 22:35:14.173020 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:35:14.172981    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b6fe05b3-4371-465c-8935-5b11c0f3a5f6-tmp-dir\") pod \"e2e-same-id-a-23ea8c7f-kserve-6c8b9765f7-s6g4p\" (UID: \"b6fe05b3-4371-465c-8935-5b11c0f3a5f6\") " pod="llm/e2e-same-id-a-23ea8c7f-kserve-6c8b9765f7-s6g4p"
Oct 30 22:35:14.173020 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:35:14.173028    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b6fe05b3-4371-465c-8935-5b11c0f3a5f6-home\") pod \"e2e-same-id-a-23ea8c7f-kserve-6c8b9765f7-s6g4p\" (UID: \"b6fe05b3-4371-465c-8935-5b11c0f3a5f6\") " pod="llm/e2e-same-id-a-23ea8c7f-kserve-6c8b9765f7-s6g4p"
Oct 30 22:35:14.173311 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:35:14.173045    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b6fe05b3-4371-465c-8935-5b11c0f3a5f6-tls-certs\") pod \"e2e-same-id-a-23ea8c7f-kserve-6c8b9765f7-s6g4p\" (UID: \"b6fe05b3-4371-465c-8935-5b11c0f3a5f6\") " pod="llm/e2e-same-id-a-23ea8c7f-kserve-6c8b9765f7-s6g4p"
Oct 30 22:35:14.173311 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:35:14.173071    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sq2m8\" (UniqueName: \"kubernetes.io/projected/b6fe05b3-4371-465c-8935-5b11c0f3a5f6-kube-api-access-sq2m8\") pod \"e2e-same-id-a-23ea8c7f-kserve-6c8b9765f7-s6g4p\" (UID: \"b6fe05b3-4371-465c-8935-5b11c0f3a5f6\") " pod="llm/e2e-same-id-a-23ea8c7f-kserve-6c8b9765f7-s6g4p"
Oct 30 22:35:14.173311 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:35:14.173109    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b6fe05b3-4371-465c-8935-5b11c0f3a5f6-dshm\") pod \"e2e-same-id-a-23ea8c7f-kserve-6c8b9765f7-s6g4p\" (UID: \"b6fe05b3-4371-465c-8935-5b11c0f3a5f6\") " pod="llm/e2e-same-id-a-23ea8c7f-kserve-6c8b9765f7-s6g4p"
Oct 30 22:35:14.173311 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:35:14.173143    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b6fe05b3-4371-465c-8935-5b11c0f3a5f6-model-cache\") pod \"e2e-same-id-a-23ea8c7f-kserve-6c8b9765f7-s6g4p\" (UID: \"b6fe05b3-4371-465c-8935-5b11c0f3a5f6\") " pod="llm/e2e-same-id-a-23ea8c7f-kserve-6c8b9765f7-s6g4p"
Oct 30 22:35:14.173533 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:35:14.173474    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b6fe05b3-4371-465c-8935-5b11c0f3a5f6-tmp-dir\") pod \"e2e-same-id-a-23ea8c7f-kserve-6c8b9765f7-s6g4p\" (UID: \"b6fe05b3-4371-465c-8935-5b11c0f3a5f6\") " pod="llm/e2e-same-id-a-23ea8c7f-kserve-6c8b9765f7-s6g4p"
Oct 30 22:35:14.173533 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:35:14.173511    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b6fe05b3-4371-465c-8935-5b11c0f3a5f6-home\") pod \"e2e-same-id-a-23ea8c7f-kserve-6c8b9765f7-s6g4p\" (UID: \"b6fe05b3-4371-465c-8935-5b11c0f3a5f6\") " pod="llm/e2e-same-id-a-23ea8c7f-kserve-6c8b9765f7-s6g4p"
Oct 30 22:35:14.173621 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:35:14.173605    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b6fe05b3-4371-465c-8935-5b11c0f3a5f6-model-cache\") pod \"e2e-same-id-a-23ea8c7f-kserve-6c8b9765f7-s6g4p\" (UID: \"b6fe05b3-4371-465c-8935-5b11c0f3a5f6\") " pod="llm/e2e-same-id-a-23ea8c7f-kserve-6c8b9765f7-s6g4p"
Oct 30 22:35:14.175705 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:35:14.175680    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b6fe05b3-4371-465c-8935-5b11c0f3a5f6-dshm\") pod \"e2e-same-id-a-23ea8c7f-kserve-6c8b9765f7-s6g4p\" (UID: \"b6fe05b3-4371-465c-8935-5b11c0f3a5f6\") " pod="llm/e2e-same-id-a-23ea8c7f-kserve-6c8b9765f7-s6g4p"
Oct 30 22:35:14.176019 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:35:14.175996    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b6fe05b3-4371-465c-8935-5b11c0f3a5f6-tls-certs\") pod \"e2e-same-id-a-23ea8c7f-kserve-6c8b9765f7-s6g4p\" (UID: \"b6fe05b3-4371-465c-8935-5b11c0f3a5f6\") " pod="llm/e2e-same-id-a-23ea8c7f-kserve-6c8b9765f7-s6g4p"
Oct 30 22:35:14.232703 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:35:14.232663    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sq2m8\" (UniqueName: \"kubernetes.io/projected/b6fe05b3-4371-465c-8935-5b11c0f3a5f6-kube-api-access-sq2m8\") pod \"e2e-same-id-a-23ea8c7f-kserve-6c8b9765f7-s6g4p\" (UID: \"b6fe05b3-4371-465c-8935-5b11c0f3a5f6\") " pod="llm/e2e-same-id-a-23ea8c7f-kserve-6c8b9765f7-s6g4p"
Oct 30 22:35:14.356124 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:35:14.356074    4473 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="983a880b-b7a6-4a47-b193-b08d5de7e75f" path="/var/lib/kubelet/pods/983a880b-b7a6-4a47-b193-b08d5de7e75f/volumes"
Oct 30 22:35:14.356560 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:35:14.356539    4473 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b2d3ef0a-6dba-482d-a6da-e5405a8bffd1" path="/var/lib/kubelet/pods/b2d3ef0a-6dba-482d-a6da-e5405a8bffd1/volumes"
Oct 30 22:35:14.513896 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:35:14.513804    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-23ea8c7f-kserve-6c8b9765f7-s6g4p"
Oct 30 22:35:14.894644 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:35:14.894614    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-23ea8c7f-kserve-6c8b9765f7-s6g4p"]
Oct 30 22:35:14.896750 ip-10-0-140-18 kubenswrapper[4473]: W1030 22:35:14.896719    4473 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb6fe05b3_4371_465c_8935_5b11c0f3a5f6.slice/crio-5c81c0b3a7642a5024bd7505452576a4f2a7c5bb635a07293d29faeee99dc42f WatchSource:0}: Error finding container 5c81c0b3a7642a5024bd7505452576a4f2a7c5bb635a07293d29faeee99dc42f: Status 404 returned error can't find the container with id 5c81c0b3a7642a5024bd7505452576a4f2a7c5bb635a07293d29faeee99dc42f
Oct 30 22:35:15.565398 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:35:15.565361    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-23ea8c7f-kserve-6c8b9765f7-s6g4p" event={"ID":"b6fe05b3-4371-465c-8935-5b11c0f3a5f6","Type":"ContainerStarted","Data":"9ad9d898d5313a4e4bb6fb25a02ba4aa2d0d8c5fe2533d49fbf372e93c01516f"}
Oct 30 22:35:15.565398 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:35:15.565404    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-23ea8c7f-kserve-6c8b9765f7-s6g4p" event={"ID":"b6fe05b3-4371-465c-8935-5b11c0f3a5f6","Type":"ContainerStarted","Data":"5c81c0b3a7642a5024bd7505452576a4f2a7c5bb635a07293d29faeee99dc42f"}
Oct 30 22:35:15.687879 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:35:15.687827    4473 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-same-id-a-23ea8c7f-kserve-6c8b9765f7-s6g4p" podStartSLOduration=2.687814037 podStartE2EDuration="2.687814037s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:35:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:35:15.686031249 +0000 UTC m=+3161.459399737" watchObservedRunningTime="2026-10-30 22:35:15.687814037 +0000 UTC m=+3161.461182502"
Oct 30 22:35:16.514825 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:35:16.514793    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-23ea8c7f-kserve-6c8b9765f7-s6g4p"
Oct 30 22:35:20.324050 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:35:20.324016    4473 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fncvg" containerName="extractor"
Oct 30 22:35:23.581151 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:35:23.581091    4473 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-5b9r4 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.146:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 22:35:23.581818 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:35:23.581765    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-5b9r4" podUID="1557635a-ac42-4385-99f3-879641eb0ccc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.146:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:35:24.514698 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:35:24.514662    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-23ea8c7f-kserve-6c8b9765f7-s6g4p"
Oct 30 22:35:24.527394 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:35:24.527363    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-23ea8c7f-kserve-6c8b9765f7-s6g4p"
Oct 30 22:35:24.631144 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:35:24.631115    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-23ea8c7f-kserve-6c8b9765f7-s6g4p"
Oct 30 22:35:50.323988 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:35:50.323888    4473 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fncvg" containerName="extractor"
Oct 30 22:35:57.871320 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:35:57.871282    4473 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-23ea8c7f-kserve-7b7d95475c-m4wgp"]
Oct 30 22:35:57.871749 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:35:57.871723    4473 cpu_manager.go:414] "containerMap: removing container" podUID="20409964-6db7-4e50-8369-c031445e2595" containerName="istio-proxy"
Oct 30 22:35:57.871749 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:35:57.871734    4473 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="20409964-6db7-4e50-8369-c031445e2595" containerName="istio-proxy"
Oct 30 22:35:57.876508 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:35:57.876490    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-23ea8c7f-kserve-7b7d95475c-m4wgp"
Oct 30 22:35:57.980474 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:35:57.980433    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-23ea8c7f-kserve-self-signed-certs\""
Oct 30 22:35:57.982675 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:35:57.982653    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-23ea8c7f-kserve-7b7d95475c-m4wgp"]
Oct 30 22:35:58.048857 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:35:58.048813    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fdc970c8-0598-4d4f-84d1-c0cd1bf053f2-model-cache\") pod \"e2e-same-id-b-23ea8c7f-kserve-7b7d95475c-m4wgp\" (UID: \"fdc970c8-0598-4d4f-84d1-c0cd1bf053f2\") " pod="llm/e2e-same-id-b-23ea8c7f-kserve-7b7d95475c-m4wgp"
Oct 30 22:35:58.049073 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:35:58.048982    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fdc970c8-0598-4d4f-84d1-c0cd1bf053f2-tls-certs\") pod \"e2e-same-id-b-23ea8c7f-kserve-7b7d95475c-m4wgp\" (UID: \"fdc970c8-0598-4d4f-84d1-c0cd1bf053f2\") " pod="llm/e2e-same-id-b-23ea8c7f-kserve-7b7d95475c-m4wgp"
Oct 30 22:35:58.049073 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:35:58.049017    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fdc970c8-0598-4d4f-84d1-c0cd1bf053f2-home\") pod \"e2e-same-id-b-23ea8c7f-kserve-7b7d95475c-m4wgp\" (UID: \"fdc970c8-0598-4d4f-84d1-c0cd1bf053f2\") " pod="llm/e2e-same-id-b-23ea8c7f-kserve-7b7d95475c-m4wgp"
Oct 30 22:35:58.049229 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:35:58.049204    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pbg7f\" (UniqueName: \"kubernetes.io/projected/fdc970c8-0598-4d4f-84d1-c0cd1bf053f2-kube-api-access-pbg7f\") pod \"e2e-same-id-b-23ea8c7f-kserve-7b7d95475c-m4wgp\" (UID: \"fdc970c8-0598-4d4f-84d1-c0cd1bf053f2\") " pod="llm/e2e-same-id-b-23ea8c7f-kserve-7b7d95475c-m4wgp"
Oct 30 22:35:58.049282 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:35:58.049257    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fdc970c8-0598-4d4f-84d1-c0cd1bf053f2-tmp-dir\") pod \"e2e-same-id-b-23ea8c7f-kserve-7b7d95475c-m4wgp\" (UID: \"fdc970c8-0598-4d4f-84d1-c0cd1bf053f2\") " pod="llm/e2e-same-id-b-23ea8c7f-kserve-7b7d95475c-m4wgp"
Oct 30 22:35:58.049345 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:35:58.049329    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fdc970c8-0598-4d4f-84d1-c0cd1bf053f2-dshm\") pod \"e2e-same-id-b-23ea8c7f-kserve-7b7d95475c-m4wgp\" (UID: \"fdc970c8-0598-4d4f-84d1-c0cd1bf053f2\") " pod="llm/e2e-same-id-b-23ea8c7f-kserve-7b7d95475c-m4wgp"
Oct 30 22:35:58.150287 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:35:58.150199    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pbg7f\" (UniqueName: \"kubernetes.io/projected/fdc970c8-0598-4d4f-84d1-c0cd1bf053f2-kube-api-access-pbg7f\") pod \"e2e-same-id-b-23ea8c7f-kserve-7b7d95475c-m4wgp\" (UID: \"fdc970c8-0598-4d4f-84d1-c0cd1bf053f2\") " pod="llm/e2e-same-id-b-23ea8c7f-kserve-7b7d95475c-m4wgp"
Oct 30 22:35:58.150287 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:35:58.150250    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fdc970c8-0598-4d4f-84d1-c0cd1bf053f2-tmp-dir\") pod \"e2e-same-id-b-23ea8c7f-kserve-7b7d95475c-m4wgp\" (UID: \"fdc970c8-0598-4d4f-84d1-c0cd1bf053f2\") " pod="llm/e2e-same-id-b-23ea8c7f-kserve-7b7d95475c-m4wgp"
Oct 30 22:35:58.150287 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:35:58.150286    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fdc970c8-0598-4d4f-84d1-c0cd1bf053f2-dshm\") pod \"e2e-same-id-b-23ea8c7f-kserve-7b7d95475c-m4wgp\" (UID: \"fdc970c8-0598-4d4f-84d1-c0cd1bf053f2\") " pod="llm/e2e-same-id-b-23ea8c7f-kserve-7b7d95475c-m4wgp"
Oct 30 22:35:58.150591 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:35:58.150333    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fdc970c8-0598-4d4f-84d1-c0cd1bf053f2-model-cache\") pod \"e2e-same-id-b-23ea8c7f-kserve-7b7d95475c-m4wgp\" (UID: \"fdc970c8-0598-4d4f-84d1-c0cd1bf053f2\") " pod="llm/e2e-same-id-b-23ea8c7f-kserve-7b7d95475c-m4wgp"
Oct 30 22:35:58.150591 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:35:58.150391    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fdc970c8-0598-4d4f-84d1-c0cd1bf053f2-tls-certs\") pod \"e2e-same-id-b-23ea8c7f-kserve-7b7d95475c-m4wgp\" (UID: \"fdc970c8-0598-4d4f-84d1-c0cd1bf053f2\") " pod="llm/e2e-same-id-b-23ea8c7f-kserve-7b7d95475c-m4wgp"
Oct 30 22:35:58.150591 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:35:58.150411    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fdc970c8-0598-4d4f-84d1-c0cd1bf053f2-home\") pod \"e2e-same-id-b-23ea8c7f-kserve-7b7d95475c-m4wgp\" (UID: \"fdc970c8-0598-4d4f-84d1-c0cd1bf053f2\") " pod="llm/e2e-same-id-b-23ea8c7f-kserve-7b7d95475c-m4wgp"
Oct 30 22:35:58.150762 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:35:58.150698    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fdc970c8-0598-4d4f-84d1-c0cd1bf053f2-tmp-dir\") pod \"e2e-same-id-b-23ea8c7f-kserve-7b7d95475c-m4wgp\" (UID: \"fdc970c8-0598-4d4f-84d1-c0cd1bf053f2\") " pod="llm/e2e-same-id-b-23ea8c7f-kserve-7b7d95475c-m4wgp"
Oct 30 22:35:58.150845 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:35:58.150809    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fdc970c8-0598-4d4f-84d1-c0cd1bf053f2-model-cache\") pod \"e2e-same-id-b-23ea8c7f-kserve-7b7d95475c-m4wgp\" (UID: \"fdc970c8-0598-4d4f-84d1-c0cd1bf053f2\") " pod="llm/e2e-same-id-b-23ea8c7f-kserve-7b7d95475c-m4wgp"
Oct 30 22:35:58.150905 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:35:58.150872    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fdc970c8-0598-4d4f-84d1-c0cd1bf053f2-home\") pod \"e2e-same-id-b-23ea8c7f-kserve-7b7d95475c-m4wgp\" (UID: \"fdc970c8-0598-4d4f-84d1-c0cd1bf053f2\") " pod="llm/e2e-same-id-b-23ea8c7f-kserve-7b7d95475c-m4wgp"
Oct 30 22:35:58.152947 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:35:58.152903    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fdc970c8-0598-4d4f-84d1-c0cd1bf053f2-dshm\") pod \"e2e-same-id-b-23ea8c7f-kserve-7b7d95475c-m4wgp\" (UID: \"fdc970c8-0598-4d4f-84d1-c0cd1bf053f2\") " pod="llm/e2e-same-id-b-23ea8c7f-kserve-7b7d95475c-m4wgp"
Oct 30 22:35:58.152947 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:35:58.152914    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fdc970c8-0598-4d4f-84d1-c0cd1bf053f2-tls-certs\") pod \"e2e-same-id-b-23ea8c7f-kserve-7b7d95475c-m4wgp\" (UID: \"fdc970c8-0598-4d4f-84d1-c0cd1bf053f2\") " pod="llm/e2e-same-id-b-23ea8c7f-kserve-7b7d95475c-m4wgp"
Oct 30 22:35:58.207526 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:35:58.207489    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pbg7f\" (UniqueName: \"kubernetes.io/projected/fdc970c8-0598-4d4f-84d1-c0cd1bf053f2-kube-api-access-pbg7f\") pod \"e2e-same-id-b-23ea8c7f-kserve-7b7d95475c-m4wgp\" (UID: \"fdc970c8-0598-4d4f-84d1-c0cd1bf053f2\") " pod="llm/e2e-same-id-b-23ea8c7f-kserve-7b7d95475c-m4wgp"
Oct 30 22:35:58.488628 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:35:58.488547    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-23ea8c7f-kserve-7b7d95475c-m4wgp"
Oct 30 22:35:58.859037 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:35:58.859010    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-23ea8c7f-kserve-7b7d95475c-m4wgp"]
Oct 30 22:35:58.861034 ip-10-0-140-18 kubenswrapper[4473]: W1030 22:35:58.861003    4473 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfdc970c8_0598_4d4f_84d1_c0cd1bf053f2.slice/crio-dc73059a973efdfe95b12a445c35e022c63adbdae2a625950206a4ac0539864f WatchSource:0}: Error finding container dc73059a973efdfe95b12a445c35e022c63adbdae2a625950206a4ac0539864f: Status 404 returned error can't find the container with id dc73059a973efdfe95b12a445c35e022c63adbdae2a625950206a4ac0539864f
Oct 30 22:35:59.788282 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:35:59.788244    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-23ea8c7f-kserve-7b7d95475c-m4wgp" event={"ID":"fdc970c8-0598-4d4f-84d1-c0cd1bf053f2","Type":"ContainerStarted","Data":"451b28ae77c3822d3e658571dff8bf145ac74b74e008d8dac6b06b52516cd98e"}
Oct 30 22:35:59.788282 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:35:59.788291    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-23ea8c7f-kserve-7b7d95475c-m4wgp" event={"ID":"fdc970c8-0598-4d4f-84d1-c0cd1bf053f2","Type":"ContainerStarted","Data":"dc73059a973efdfe95b12a445c35e022c63adbdae2a625950206a4ac0539864f"}
Oct 30 22:35:59.903845 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:35:59.903786    4473 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-same-id-b-23ea8c7f-kserve-7b7d95475c-m4wgp" podStartSLOduration=2.903769661 podStartE2EDuration="2.903769661s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:35:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:35:59.902473258 +0000 UTC m=+3205.675841744" watchObservedRunningTime="2026-10-30 22:35:59.903769661 +0000 UTC m=+3205.677138147"
Oct 30 22:36:00.489327 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:36:00.489290    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-23ea8c7f-kserve-7b7d95475c-m4wgp"
Oct 30 22:36:08.489661 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:36:08.489620    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-23ea8c7f-kserve-7b7d95475c-m4wgp"
Oct 30 22:36:08.502620 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:36:08.502590    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-23ea8c7f-kserve-7b7d95475c-m4wgp"
Oct 30 22:36:08.864865 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:36:08.864837    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-23ea8c7f-kserve-7b7d95475c-m4wgp"
Oct 30 22:36:20.324087 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:36:20.324049    4473 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fncvg" containerName="extractor"
Oct 30 22:36:31.845450 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:36:31.845406    4473 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-zgjtb container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.98:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 22:36:31.846170 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:36:31.846134    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-zgjtb" podUID="a13c1563-5b04-424a-b761-62a4b85f16e7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.98:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:36:40.510719 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:36:40.510679    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-23ea8c7f-kserve-6c8b9765f7-s6g4p"]
Oct 30 22:36:45.134662 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:36:45.134618    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-23ea8c7f-kserve-7b7d95475c-m4wgp"]
Oct 30 22:36:50.323186 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:36:50.323147    4473 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fncvg" containerName="extractor"
Oct 30 22:36:53.577089 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:36:53.577043    4473 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-5b9r4 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.146:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 22:36:53.577740 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:36:53.577708    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-5b9r4" podUID="1557635a-ac42-4385-99f3-879641eb0ccc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.146:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:36:55.531080 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:36:55.531019    4473 kuberuntime_container.go:898] "Killing container with a grace period" pod="llm/e2e-same-id-a-23ea8c7f-kserve-6c8b9765f7-s6g4p" podUID="b6fe05b3-4371-465c-8935-5b11c0f3a5f6" containerName="main" containerID="cri-o://9ad9d898d5313a4e4bb6fb25a02ba4aa2d0d8c5fe2533d49fbf372e93c01516f" gracePeriod=45
Oct 30 22:36:55.621050 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:36:55.621009    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/e2e-same-id-a-23ea8c7f-kserve-6c8b9765f7-s6g4p" podUID="b6fe05b3-4371-465c-8935-5b11c0f3a5f6" containerName="main" probeResult="failure" output="Get \"https://10.133.0.160:8000/ready\": dial tcp 10.133.0.160:8000: connect: connection refused"
Oct 30 22:36:55.779998 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:36:55.779970    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-23ea8c7f-kserve-6c8b9765f7-s6g4p"
Oct 30 22:36:55.857819 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:36:55.857739    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b6fe05b3-4371-465c-8935-5b11c0f3a5f6-dshm\" (UniqueName: \"kubernetes.io/empty-dir/b6fe05b3-4371-465c-8935-5b11c0f3a5f6-dshm\") pod \"b6fe05b3-4371-465c-8935-5b11c0f3a5f6\" (UID: \"b6fe05b3-4371-465c-8935-5b11c0f3a5f6\") "
Oct 30 22:36:55.857819 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:36:55.857784    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b6fe05b3-4371-465c-8935-5b11c0f3a5f6-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b6fe05b3-4371-465c-8935-5b11c0f3a5f6-tmp-dir\") pod \"b6fe05b3-4371-465c-8935-5b11c0f3a5f6\" (UID: \"b6fe05b3-4371-465c-8935-5b11c0f3a5f6\") "
Oct 30 22:36:55.858106 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:36:55.857850    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b6fe05b3-4371-465c-8935-5b11c0f3a5f6-kube-api-access-sq2m8\" (UniqueName: \"kubernetes.io/projected/b6fe05b3-4371-465c-8935-5b11c0f3a5f6-kube-api-access-sq2m8\") pod \"b6fe05b3-4371-465c-8935-5b11c0f3a5f6\" (UID: \"b6fe05b3-4371-465c-8935-5b11c0f3a5f6\") "
Oct 30 22:36:55.858106 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:36:55.857873    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b6fe05b3-4371-465c-8935-5b11c0f3a5f6-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b6fe05b3-4371-465c-8935-5b11c0f3a5f6-model-cache\") pod \"b6fe05b3-4371-465c-8935-5b11c0f3a5f6\" (UID: \"b6fe05b3-4371-465c-8935-5b11c0f3a5f6\") "
Oct 30 22:36:55.858106 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:36:55.858001    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b6fe05b3-4371-465c-8935-5b11c0f3a5f6-home\" (UniqueName: \"kubernetes.io/empty-dir/b6fe05b3-4371-465c-8935-5b11c0f3a5f6-home\") pod \"b6fe05b3-4371-465c-8935-5b11c0f3a5f6\" (UID: \"b6fe05b3-4371-465c-8935-5b11c0f3a5f6\") "
Oct 30 22:36:55.858106 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:36:55.858041    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b6fe05b3-4371-465c-8935-5b11c0f3a5f6-tls-certs\" (UniqueName: \"kubernetes.io/secret/b6fe05b3-4371-465c-8935-5b11c0f3a5f6-tls-certs\") pod \"b6fe05b3-4371-465c-8935-5b11c0f3a5f6\" (UID: \"b6fe05b3-4371-465c-8935-5b11c0f3a5f6\") "
Oct 30 22:36:55.858334 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:36:55.858137    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b6fe05b3-4371-465c-8935-5b11c0f3a5f6-tmp-dir" pod "b6fe05b3-4371-465c-8935-5b11c0f3a5f6" (UID: "b6fe05b3-4371-465c-8935-5b11c0f3a5f6"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:36:55.858334 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:36:55.858194    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b6fe05b3-4371-465c-8935-5b11c0f3a5f6-model-cache" pod "b6fe05b3-4371-465c-8935-5b11c0f3a5f6" (UID: "b6fe05b3-4371-465c-8935-5b11c0f3a5f6"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:36:55.858446 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:36:55.858413    4473 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b6fe05b3-4371-465c-8935-5b11c0f3a5f6-tmp-dir\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:36:55.858446 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:36:55.858434    4473 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b6fe05b3-4371-465c-8935-5b11c0f3a5f6-model-cache\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:36:55.858577 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:36:55.858553    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b6fe05b3-4371-465c-8935-5b11c0f3a5f6-home" pod "b6fe05b3-4371-465c-8935-5b11c0f3a5f6" (UID: "b6fe05b3-4371-465c-8935-5b11c0f3a5f6"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:36:55.860316 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:36:55.860284    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b6fe05b3-4371-465c-8935-5b11c0f3a5f6-kube-api-access-sq2m8" pod "b6fe05b3-4371-465c-8935-5b11c0f3a5f6" (UID: "b6fe05b3-4371-465c-8935-5b11c0f3a5f6"). InnerVolumeSpecName "kube-api-access-sq2m8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:36:55.860316 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:36:55.860303    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b6fe05b3-4371-465c-8935-5b11c0f3a5f6-dshm" pod "b6fe05b3-4371-465c-8935-5b11c0f3a5f6" (UID: "b6fe05b3-4371-465c-8935-5b11c0f3a5f6"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:36:55.860522 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:36:55.860328    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b6fe05b3-4371-465c-8935-5b11c0f3a5f6-tls-certs" pod "b6fe05b3-4371-465c-8935-5b11c0f3a5f6" (UID: "b6fe05b3-4371-465c-8935-5b11c0f3a5f6"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:36:55.959500 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:36:55.959470    4473 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b6fe05b3-4371-465c-8935-5b11c0f3a5f6-home\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:36:55.959500 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:36:55.959498    4473 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b6fe05b3-4371-465c-8935-5b11c0f3a5f6-tls-certs\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:36:55.959500 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:36:55.959507    4473 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b6fe05b3-4371-465c-8935-5b11c0f3a5f6-dshm\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:36:55.959742 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:36:55.959515    4473 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sq2m8\" (UniqueName: \"kubernetes.io/projected/b6fe05b3-4371-465c-8935-5b11c0f3a5f6-kube-api-access-sq2m8\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:36:56.277291 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:36:56.277257    4473 generic.go:356] "Generic (PLEG): container finished" podID="b6fe05b3-4371-465c-8935-5b11c0f3a5f6" containerID="9ad9d898d5313a4e4bb6fb25a02ba4aa2d0d8c5fe2533d49fbf372e93c01516f" exitCode=0
Oct 30 22:36:56.277504 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:36:56.277313    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-23ea8c7f-kserve-6c8b9765f7-s6g4p"
Oct 30 22:36:56.277504 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:36:56.277346    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-23ea8c7f-kserve-6c8b9765f7-s6g4p" event={"ID":"b6fe05b3-4371-465c-8935-5b11c0f3a5f6","Type":"ContainerDied","Data":"9ad9d898d5313a4e4bb6fb25a02ba4aa2d0d8c5fe2533d49fbf372e93c01516f"}
Oct 30 22:36:56.277504 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:36:56.277395    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-23ea8c7f-kserve-6c8b9765f7-s6g4p" event={"ID":"b6fe05b3-4371-465c-8935-5b11c0f3a5f6","Type":"ContainerDied","Data":"5c81c0b3a7642a5024bd7505452576a4f2a7c5bb635a07293d29faeee99dc42f"}
Oct 30 22:36:56.277504 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:36:56.277412    4473 scope.go:122] "RemoveContainer" containerID="9ad9d898d5313a4e4bb6fb25a02ba4aa2d0d8c5fe2533d49fbf372e93c01516f"
Oct 30 22:36:56.287133 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:36:56.287114    4473 scope.go:122] "RemoveContainer" containerID="9ad9d898d5313a4e4bb6fb25a02ba4aa2d0d8c5fe2533d49fbf372e93c01516f"
Oct 30 22:36:56.287457 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:36:56.287435    4473 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9ad9d898d5313a4e4bb6fb25a02ba4aa2d0d8c5fe2533d49fbf372e93c01516f\": container with ID starting with 9ad9d898d5313a4e4bb6fb25a02ba4aa2d0d8c5fe2533d49fbf372e93c01516f not found: ID does not exist" containerID="9ad9d898d5313a4e4bb6fb25a02ba4aa2d0d8c5fe2533d49fbf372e93c01516f"
Oct 30 22:36:56.287511 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:36:56.287467    4473 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9ad9d898d5313a4e4bb6fb25a02ba4aa2d0d8c5fe2533d49fbf372e93c01516f"} err="failed to get container status \"9ad9d898d5313a4e4bb6fb25a02ba4aa2d0d8c5fe2533d49fbf372e93c01516f\": rpc error: code = NotFound desc = could not find container \"9ad9d898d5313a4e4bb6fb25a02ba4aa2d0d8c5fe2533d49fbf372e93c01516f\": container with ID starting with 9ad9d898d5313a4e4bb6fb25a02ba4aa2d0d8c5fe2533d49fbf372e93c01516f not found: ID does not exist"
Oct 30 22:36:56.465648 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:36:56.465613    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-23ea8c7f-kserve-6c8b9765f7-s6g4p"]
Oct 30 22:36:56.515552 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:36:56.515517    4473 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-23ea8c7f-kserve-6c8b9765f7-s6g4p"]
Oct 30 22:36:58.355891 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:36:58.355861    4473 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b6fe05b3-4371-465c-8935-5b11c0f3a5f6" path="/var/lib/kubelet/pods/b6fe05b3-4371-465c-8935-5b11c0f3a5f6/volumes"
Oct 30 22:37:00.166612 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:00.166548    4473 kuberuntime_container.go:898] "Killing container with a grace period" pod="llm/e2e-same-id-b-23ea8c7f-kserve-7b7d95475c-m4wgp" podUID="fdc970c8-0598-4d4f-84d1-c0cd1bf053f2" containerName="main" containerID="cri-o://451b28ae77c3822d3e658571dff8bf145ac74b74e008d8dac6b06b52516cd98e" gracePeriod=45
Oct 30 22:37:00.300277 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:00.300250    4473 generic.go:356] "Generic (PLEG): container finished" podID="fdc970c8-0598-4d4f-84d1-c0cd1bf053f2" containerID="451b28ae77c3822d3e658571dff8bf145ac74b74e008d8dac6b06b52516cd98e" exitCode=0
Oct 30 22:37:00.300277 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:00.300288    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-23ea8c7f-kserve-7b7d95475c-m4wgp" event={"ID":"fdc970c8-0598-4d4f-84d1-c0cd1bf053f2","Type":"ContainerDied","Data":"451b28ae77c3822d3e658571dff8bf145ac74b74e008d8dac6b06b52516cd98e"}
Oct 30 22:37:00.416085 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:00.416060    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-23ea8c7f-kserve-7b7d95475c-m4wgp"
Oct 30 22:37:00.500899 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:00.500812    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fdc970c8-0598-4d4f-84d1-c0cd1bf053f2-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fdc970c8-0598-4d4f-84d1-c0cd1bf053f2-tmp-dir\") pod \"fdc970c8-0598-4d4f-84d1-c0cd1bf053f2\" (UID: \"fdc970c8-0598-4d4f-84d1-c0cd1bf053f2\") "
Oct 30 22:37:00.500899 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:00.500871    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fdc970c8-0598-4d4f-84d1-c0cd1bf053f2-kube-api-access-pbg7f\" (UniqueName: \"kubernetes.io/projected/fdc970c8-0598-4d4f-84d1-c0cd1bf053f2-kube-api-access-pbg7f\") pod \"fdc970c8-0598-4d4f-84d1-c0cd1bf053f2\" (UID: \"fdc970c8-0598-4d4f-84d1-c0cd1bf053f2\") "
Oct 30 22:37:00.501183 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:00.500934    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/fdc970c8-0598-4d4f-84d1-c0cd1bf053f2-tls-certs\" (UniqueName: \"kubernetes.io/secret/fdc970c8-0598-4d4f-84d1-c0cd1bf053f2-tls-certs\") pod \"fdc970c8-0598-4d4f-84d1-c0cd1bf053f2\" (UID: \"fdc970c8-0598-4d4f-84d1-c0cd1bf053f2\") "
Oct 30 22:37:00.501183 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:00.501026    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fdc970c8-0598-4d4f-84d1-c0cd1bf053f2-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fdc970c8-0598-4d4f-84d1-c0cd1bf053f2-model-cache\") pod \"fdc970c8-0598-4d4f-84d1-c0cd1bf053f2\" (UID: \"fdc970c8-0598-4d4f-84d1-c0cd1bf053f2\") "
Oct 30 22:37:00.501183 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:00.501109    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fdc970c8-0598-4d4f-84d1-c0cd1bf053f2-home\" (UniqueName: \"kubernetes.io/empty-dir/fdc970c8-0598-4d4f-84d1-c0cd1bf053f2-home\") pod \"fdc970c8-0598-4d4f-84d1-c0cd1bf053f2\" (UID: \"fdc970c8-0598-4d4f-84d1-c0cd1bf053f2\") "
Oct 30 22:37:00.501183 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:00.501134    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fdc970c8-0598-4d4f-84d1-c0cd1bf053f2-tmp-dir" pod "fdc970c8-0598-4d4f-84d1-c0cd1bf053f2" (UID: "fdc970c8-0598-4d4f-84d1-c0cd1bf053f2"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:37:00.501183 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:00.501154    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fdc970c8-0598-4d4f-84d1-c0cd1bf053f2-dshm\" (UniqueName: \"kubernetes.io/empty-dir/fdc970c8-0598-4d4f-84d1-c0cd1bf053f2-dshm\") pod \"fdc970c8-0598-4d4f-84d1-c0cd1bf053f2\" (UID: \"fdc970c8-0598-4d4f-84d1-c0cd1bf053f2\") "
Oct 30 22:37:00.501432 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:00.501327    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fdc970c8-0598-4d4f-84d1-c0cd1bf053f2-home" pod "fdc970c8-0598-4d4f-84d1-c0cd1bf053f2" (UID: "fdc970c8-0598-4d4f-84d1-c0cd1bf053f2"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:37:00.501432 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:00.501339    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fdc970c8-0598-4d4f-84d1-c0cd1bf053f2-model-cache" pod "fdc970c8-0598-4d4f-84d1-c0cd1bf053f2" (UID: "fdc970c8-0598-4d4f-84d1-c0cd1bf053f2"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:37:00.501524 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:00.501498    4473 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fdc970c8-0598-4d4f-84d1-c0cd1bf053f2-model-cache\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:37:00.501524 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:00.501512    4473 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fdc970c8-0598-4d4f-84d1-c0cd1bf053f2-home\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:37:00.501524 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:00.501521    4473 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fdc970c8-0598-4d4f-84d1-c0cd1bf053f2-tmp-dir\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:37:00.503368 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:00.503345    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fdc970c8-0598-4d4f-84d1-c0cd1bf053f2-dshm" pod "fdc970c8-0598-4d4f-84d1-c0cd1bf053f2" (UID: "fdc970c8-0598-4d4f-84d1-c0cd1bf053f2"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:37:00.503718 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:00.503699    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fdc970c8-0598-4d4f-84d1-c0cd1bf053f2-tls-certs" pod "fdc970c8-0598-4d4f-84d1-c0cd1bf053f2" (UID: "fdc970c8-0598-4d4f-84d1-c0cd1bf053f2"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:37:00.503781 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:00.503725    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fdc970c8-0598-4d4f-84d1-c0cd1bf053f2-kube-api-access-pbg7f" pod "fdc970c8-0598-4d4f-84d1-c0cd1bf053f2" (UID: "fdc970c8-0598-4d4f-84d1-c0cd1bf053f2"). InnerVolumeSpecName "kube-api-access-pbg7f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:37:00.602250 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:00.602208    4473 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fdc970c8-0598-4d4f-84d1-c0cd1bf053f2-dshm\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:37:00.602250 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:00.602246    4473 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pbg7f\" (UniqueName: \"kubernetes.io/projected/fdc970c8-0598-4d4f-84d1-c0cd1bf053f2-kube-api-access-pbg7f\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:37:00.602250 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:00.602260    4473 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fdc970c8-0598-4d4f-84d1-c0cd1bf053f2-tls-certs\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:37:01.306317 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:01.306288    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-23ea8c7f-kserve-7b7d95475c-m4wgp"
Oct 30 22:37:01.306750 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:01.306319    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-23ea8c7f-kserve-7b7d95475c-m4wgp" event={"ID":"fdc970c8-0598-4d4f-84d1-c0cd1bf053f2","Type":"ContainerDied","Data":"dc73059a973efdfe95b12a445c35e022c63adbdae2a625950206a4ac0539864f"}
Oct 30 22:37:01.306750 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:01.306366    4473 scope.go:122] "RemoveContainer" containerID="451b28ae77c3822d3e658571dff8bf145ac74b74e008d8dac6b06b52516cd98e"
Oct 30 22:37:01.472392 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:01.472356    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-23ea8c7f-kserve-7b7d95475c-m4wgp"]
Oct 30 22:37:01.475574 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:01.475547    4473 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-23ea8c7f-kserve-7b7d95475c-m4wgp"]
Oct 30 22:37:02.355678 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:02.355642    4473 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fdc970c8-0598-4d4f-84d1-c0cd1bf053f2" path="/var/lib/kubelet/pods/fdc970c8-0598-4d4f-84d1-c0cd1bf053f2/volumes"
Oct 30 22:37:20.323334 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:37:20.323248    4473 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fncvg" containerName="extractor"
Oct 30 22:37:31.899803 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:31.899767    4473 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-64979f7667-qkx6z"]
Oct 30 22:37:31.900360 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:31.900197    4473 cpu_manager.go:414] "containerMap: removing container" podUID="fdc970c8-0598-4d4f-84d1-c0cd1bf053f2" containerName="main"
Oct 30 22:37:31.900360 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:31.900208    4473 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fdc970c8-0598-4d4f-84d1-c0cd1bf053f2" containerName="main"
Oct 30 22:37:31.900360 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:31.900236    4473 cpu_manager.go:414] "containerMap: removing container" podUID="b6fe05b3-4371-465c-8935-5b11c0f3a5f6" containerName="main"
Oct 30 22:37:31.900360 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:31.900242    4473 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b6fe05b3-4371-465c-8935-5b11c0f3a5f6" containerName="main"
Oct 30 22:37:31.900360 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:31.900306    4473 memory_manager.go:365] "RemoveStaleState removing state" podUID="b6fe05b3-4371-465c-8935-5b11c0f3a5f6" containerName="main"
Oct 30 22:37:31.900360 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:31.900316    4473 memory_manager.go:365] "RemoveStaleState removing state" podUID="fdc970c8-0598-4d4f-84d1-c0cd1bf053f2" containerName="main"
Oct 30 22:37:31.903216 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:31.903199    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-64979f7667-qkx6z"
Oct 30 22:37:32.012152 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:32.012109    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7c8203e1-57fc-4227-91b6-769ce42af364-maas-api-tls\") pod \"maas-api-64979f7667-qkx6z\" (UID: \"7c8203e1-57fc-4227-91b6-769ce42af364\") " pod="odh-ai-gateway-infra/maas-api-64979f7667-qkx6z"
Oct 30 22:37:32.012383 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:32.012186    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7c8203e1-57fc-4227-91b6-769ce42af364-metrics-certs\") pod \"maas-api-64979f7667-qkx6z\" (UID: \"7c8203e1-57fc-4227-91b6-769ce42af364\") " pod="odh-ai-gateway-infra/maas-api-64979f7667-qkx6z"
Oct 30 22:37:32.012383 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:32.012218    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9jd7w\" (UniqueName: \"kubernetes.io/projected/7c8203e1-57fc-4227-91b6-769ce42af364-kube-api-access-9jd7w\") pod \"maas-api-64979f7667-qkx6z\" (UID: \"7c8203e1-57fc-4227-91b6-769ce42af364\") " pod="odh-ai-gateway-infra/maas-api-64979f7667-qkx6z"
Oct 30 22:37:32.012383 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:32.012350    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7c8203e1-57fc-4227-91b6-769ce42af364-openshift-service-ca\") pod \"maas-api-64979f7667-qkx6z\" (UID: \"7c8203e1-57fc-4227-91b6-769ce42af364\") " pod="odh-ai-gateway-infra/maas-api-64979f7667-qkx6z"
Oct 30 22:37:32.014203 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:32.014133    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-64979f7667-qkx6z"]
Oct 30 22:37:32.113814 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:32.113776    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7c8203e1-57fc-4227-91b6-769ce42af364-metrics-certs\") pod \"maas-api-64979f7667-qkx6z\" (UID: \"7c8203e1-57fc-4227-91b6-769ce42af364\") " pod="odh-ai-gateway-infra/maas-api-64979f7667-qkx6z"
Oct 30 22:37:32.113814 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:32.113822    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9jd7w\" (UniqueName: \"kubernetes.io/projected/7c8203e1-57fc-4227-91b6-769ce42af364-kube-api-access-9jd7w\") pod \"maas-api-64979f7667-qkx6z\" (UID: \"7c8203e1-57fc-4227-91b6-769ce42af364\") " pod="odh-ai-gateway-infra/maas-api-64979f7667-qkx6z"
Oct 30 22:37:32.114151 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:32.113952    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7c8203e1-57fc-4227-91b6-769ce42af364-openshift-service-ca\") pod \"maas-api-64979f7667-qkx6z\" (UID: \"7c8203e1-57fc-4227-91b6-769ce42af364\") " pod="odh-ai-gateway-infra/maas-api-64979f7667-qkx6z"
Oct 30 22:37:32.114151 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:32.114000    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7c8203e1-57fc-4227-91b6-769ce42af364-maas-api-tls\") pod \"maas-api-64979f7667-qkx6z\" (UID: \"7c8203e1-57fc-4227-91b6-769ce42af364\") " pod="odh-ai-gateway-infra/maas-api-64979f7667-qkx6z"
Oct 30 22:37:32.114670 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:32.114644    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7c8203e1-57fc-4227-91b6-769ce42af364-openshift-service-ca\") pod \"maas-api-64979f7667-qkx6z\" (UID: \"7c8203e1-57fc-4227-91b6-769ce42af364\") " pod="odh-ai-gateway-infra/maas-api-64979f7667-qkx6z"
Oct 30 22:37:32.116475 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:32.116453    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7c8203e1-57fc-4227-91b6-769ce42af364-metrics-certs\") pod \"maas-api-64979f7667-qkx6z\" (UID: \"7c8203e1-57fc-4227-91b6-769ce42af364\") " pod="odh-ai-gateway-infra/maas-api-64979f7667-qkx6z"
Oct 30 22:37:32.116581 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:32.116520    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7c8203e1-57fc-4227-91b6-769ce42af364-maas-api-tls\") pod \"maas-api-64979f7667-qkx6z\" (UID: \"7c8203e1-57fc-4227-91b6-769ce42af364\") " pod="odh-ai-gateway-infra/maas-api-64979f7667-qkx6z"
Oct 30 22:37:32.171827 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:32.171747    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9jd7w\" (UniqueName: \"kubernetes.io/projected/7c8203e1-57fc-4227-91b6-769ce42af364-kube-api-access-9jd7w\") pod \"maas-api-64979f7667-qkx6z\" (UID: \"7c8203e1-57fc-4227-91b6-769ce42af364\") " pod="odh-ai-gateway-infra/maas-api-64979f7667-qkx6z"
Oct 30 22:37:32.223657 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:32.223623    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-64979f7667-qkx6z"
Oct 30 22:37:32.594586 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:32.594560    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-64979f7667-qkx6z"]
Oct 30 22:37:32.596559 ip-10-0-140-18 kubenswrapper[4473]: W1030 22:37:32.596529    4473 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c8203e1_57fc_4227_91b6_769ce42af364.slice/crio-6cf9d8618d9b7d25b03f57041c1e09733d9b915843d8b64a171523f1a88a249f WatchSource:0}: Error finding container 6cf9d8618d9b7d25b03f57041c1e09733d9b915843d8b64a171523f1a88a249f: Status 404 returned error can't find the container with id 6cf9d8618d9b7d25b03f57041c1e09733d9b915843d8b64a171523f1a88a249f
Oct 30 22:37:32.597896 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:32.597875    4473 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 22:37:33.505499 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:33.505460    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-64979f7667-qkx6z" event={"ID":"7c8203e1-57fc-4227-91b6-769ce42af364","Type":"ContainerStarted","Data":"6cf9d8618d9b7d25b03f57041c1e09733d9b915843d8b64a171523f1a88a249f"}
Oct 30 22:37:34.511557 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:34.511518    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-64979f7667-qkx6z" event={"ID":"7c8203e1-57fc-4227-91b6-769ce42af364","Type":"ContainerStarted","Data":"c74cd0607fb0239acda0ab7208a4bb5e343453142782129c4843c9504af29fa7"}
Oct 30 22:37:34.512014 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:34.511700    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-64979f7667-qkx6z"
Oct 30 22:37:34.629023 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:34.628967    4473 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-64979f7667-qkx6z" podStartSLOduration=2.658433499 podStartE2EDuration="3.628951092s" totalImagesPullingTime="970.517593ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:37:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:37:34.628359458 +0000 UTC m=+3300.401727947" watchObservedRunningTime="2026-10-30 22:37:34.628951092 +0000 UTC m=+3300.402319573"
Oct 30 22:37:38.868325 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:38.868293    4473 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-f56a3efc-openshift-default-588cd5c7c5-qpzwv"]
Oct 30 22:37:38.873492 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:38.873471    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-f56a3efc-openshift-default-588cd5c7c5-qpzwv"
Oct 30 22:37:38.879619 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:38.879593    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/673eabe0-3a0e-4b7e-82f3-1653e4557d5c-istio-token\") pod \"e2e-ait-f56a3efc-openshift-default-588cd5c7c5-qpzwv\" (UID: \"673eabe0-3a0e-4b7e-82f3-1653e4557d5c\") " pod="openshift-ingress/e2e-ait-f56a3efc-openshift-default-588cd5c7c5-qpzwv"
Oct 30 22:37:38.879749 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:38.879627    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/673eabe0-3a0e-4b7e-82f3-1653e4557d5c-workload-socket\") pod \"e2e-ait-f56a3efc-openshift-default-588cd5c7c5-qpzwv\" (UID: \"673eabe0-3a0e-4b7e-82f3-1653e4557d5c\") " pod="openshift-ingress/e2e-ait-f56a3efc-openshift-default-588cd5c7c5-qpzwv"
Oct 30 22:37:38.879749 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:38.879652    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/673eabe0-3a0e-4b7e-82f3-1653e4557d5c-credential-socket\") pod \"e2e-ait-f56a3efc-openshift-default-588cd5c7c5-qpzwv\" (UID: \"673eabe0-3a0e-4b7e-82f3-1653e4557d5c\") " pod="openshift-ingress/e2e-ait-f56a3efc-openshift-default-588cd5c7c5-qpzwv"
Oct 30 22:37:38.879749 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:38.879740    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/673eabe0-3a0e-4b7e-82f3-1653e4557d5c-workload-certs\") pod \"e2e-ait-f56a3efc-openshift-default-588cd5c7c5-qpzwv\" (UID: \"673eabe0-3a0e-4b7e-82f3-1653e4557d5c\") " pod="openshift-ingress/e2e-ait-f56a3efc-openshift-default-588cd5c7c5-qpzwv"
Oct 30 22:37:38.879875 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:38.879766    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/673eabe0-3a0e-4b7e-82f3-1653e4557d5c-istio-data\") pod \"e2e-ait-f56a3efc-openshift-default-588cd5c7c5-qpzwv\" (UID: \"673eabe0-3a0e-4b7e-82f3-1653e4557d5c\") " pod="openshift-ingress/e2e-ait-f56a3efc-openshift-default-588cd5c7c5-qpzwv"
Oct 30 22:37:38.879875 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:38.879808    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/673eabe0-3a0e-4b7e-82f3-1653e4557d5c-istiod-ca-cert\") pod \"e2e-ait-f56a3efc-openshift-default-588cd5c7c5-qpzwv\" (UID: \"673eabe0-3a0e-4b7e-82f3-1653e4557d5c\") " pod="openshift-ingress/e2e-ait-f56a3efc-openshift-default-588cd5c7c5-qpzwv"
Oct 30 22:37:38.880009 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:38.879990    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/673eabe0-3a0e-4b7e-82f3-1653e4557d5c-istio-envoy\") pod \"e2e-ait-f56a3efc-openshift-default-588cd5c7c5-qpzwv\" (UID: \"673eabe0-3a0e-4b7e-82f3-1653e4557d5c\") " pod="openshift-ingress/e2e-ait-f56a3efc-openshift-default-588cd5c7c5-qpzwv"
Oct 30 22:37:38.880068 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:38.880043    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pntsd\" (UniqueName: \"kubernetes.io/projected/673eabe0-3a0e-4b7e-82f3-1653e4557d5c-kube-api-access-pntsd\") pod \"e2e-ait-f56a3efc-openshift-default-588cd5c7c5-qpzwv\" (UID: \"673eabe0-3a0e-4b7e-82f3-1653e4557d5c\") " pod="openshift-ingress/e2e-ait-f56a3efc-openshift-default-588cd5c7c5-qpzwv"
Oct 30 22:37:38.880128 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:38.880086    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/673eabe0-3a0e-4b7e-82f3-1653e4557d5c-istio-podinfo\") pod \"e2e-ait-f56a3efc-openshift-default-588cd5c7c5-qpzwv\" (UID: \"673eabe0-3a0e-4b7e-82f3-1653e4557d5c\") " pod="openshift-ingress/e2e-ait-f56a3efc-openshift-default-588cd5c7c5-qpzwv"
Oct 30 22:37:38.926412 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:38.926381    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-f56a3efc-openshift-default-dockercfg-gqrjb\""
Oct 30 22:37:38.979902 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:38.979866    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-f56a3efc-openshift-default-588cd5c7c5-qpzwv"]
Oct 30 22:37:38.981495 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:38.981458    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/673eabe0-3a0e-4b7e-82f3-1653e4557d5c-workload-certs\") pod \"e2e-ait-f56a3efc-openshift-default-588cd5c7c5-qpzwv\" (UID: \"673eabe0-3a0e-4b7e-82f3-1653e4557d5c\") " pod="openshift-ingress/e2e-ait-f56a3efc-openshift-default-588cd5c7c5-qpzwv"
Oct 30 22:37:38.981670 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:38.981509    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/673eabe0-3a0e-4b7e-82f3-1653e4557d5c-istio-data\") pod \"e2e-ait-f56a3efc-openshift-default-588cd5c7c5-qpzwv\" (UID: \"673eabe0-3a0e-4b7e-82f3-1653e4557d5c\") " pod="openshift-ingress/e2e-ait-f56a3efc-openshift-default-588cd5c7c5-qpzwv"
Oct 30 22:37:38.981670 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:38.981561    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/673eabe0-3a0e-4b7e-82f3-1653e4557d5c-istiod-ca-cert\") pod \"e2e-ait-f56a3efc-openshift-default-588cd5c7c5-qpzwv\" (UID: \"673eabe0-3a0e-4b7e-82f3-1653e4557d5c\") " pod="openshift-ingress/e2e-ait-f56a3efc-openshift-default-588cd5c7c5-qpzwv"
Oct 30 22:37:38.981758 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:38.981684    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/673eabe0-3a0e-4b7e-82f3-1653e4557d5c-istio-envoy\") pod \"e2e-ait-f56a3efc-openshift-default-588cd5c7c5-qpzwv\" (UID: \"673eabe0-3a0e-4b7e-82f3-1653e4557d5c\") " pod="openshift-ingress/e2e-ait-f56a3efc-openshift-default-588cd5c7c5-qpzwv"
Oct 30 22:37:38.981758 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:38.981727    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pntsd\" (UniqueName: \"kubernetes.io/projected/673eabe0-3a0e-4b7e-82f3-1653e4557d5c-kube-api-access-pntsd\") pod \"e2e-ait-f56a3efc-openshift-default-588cd5c7c5-qpzwv\" (UID: \"673eabe0-3a0e-4b7e-82f3-1653e4557d5c\") " pod="openshift-ingress/e2e-ait-f56a3efc-openshift-default-588cd5c7c5-qpzwv"
Oct 30 22:37:38.981956 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:38.981768    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/673eabe0-3a0e-4b7e-82f3-1653e4557d5c-istio-podinfo\") pod \"e2e-ait-f56a3efc-openshift-default-588cd5c7c5-qpzwv\" (UID: \"673eabe0-3a0e-4b7e-82f3-1653e4557d5c\") " pod="openshift-ingress/e2e-ait-f56a3efc-openshift-default-588cd5c7c5-qpzwv"
Oct 30 22:37:38.981956 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:38.981837    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/673eabe0-3a0e-4b7e-82f3-1653e4557d5c-istio-token\") pod \"e2e-ait-f56a3efc-openshift-default-588cd5c7c5-qpzwv\" (UID: \"673eabe0-3a0e-4b7e-82f3-1653e4557d5c\") " pod="openshift-ingress/e2e-ait-f56a3efc-openshift-default-588cd5c7c5-qpzwv"
Oct 30 22:37:38.981956 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:38.981866    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/673eabe0-3a0e-4b7e-82f3-1653e4557d5c-workload-socket\") pod \"e2e-ait-f56a3efc-openshift-default-588cd5c7c5-qpzwv\" (UID: \"673eabe0-3a0e-4b7e-82f3-1653e4557d5c\") " pod="openshift-ingress/e2e-ait-f56a3efc-openshift-default-588cd5c7c5-qpzwv"
Oct 30 22:37:38.981956 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:38.981891    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/673eabe0-3a0e-4b7e-82f3-1653e4557d5c-workload-certs\") pod \"e2e-ait-f56a3efc-openshift-default-588cd5c7c5-qpzwv\" (UID: \"673eabe0-3a0e-4b7e-82f3-1653e4557d5c\") " pod="openshift-ingress/e2e-ait-f56a3efc-openshift-default-588cd5c7c5-qpzwv"
Oct 30 22:37:38.981956 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:38.981899    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/673eabe0-3a0e-4b7e-82f3-1653e4557d5c-credential-socket\") pod \"e2e-ait-f56a3efc-openshift-default-588cd5c7c5-qpzwv\" (UID: \"673eabe0-3a0e-4b7e-82f3-1653e4557d5c\") " pod="openshift-ingress/e2e-ait-f56a3efc-openshift-default-588cd5c7c5-qpzwv"
Oct 30 22:37:38.982243 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:38.981987    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/673eabe0-3a0e-4b7e-82f3-1653e4557d5c-istio-data\") pod \"e2e-ait-f56a3efc-openshift-default-588cd5c7c5-qpzwv\" (UID: \"673eabe0-3a0e-4b7e-82f3-1653e4557d5c\") " pod="openshift-ingress/e2e-ait-f56a3efc-openshift-default-588cd5c7c5-qpzwv"
Oct 30 22:37:38.982243 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:38.982227    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/673eabe0-3a0e-4b7e-82f3-1653e4557d5c-credential-socket\") pod \"e2e-ait-f56a3efc-openshift-default-588cd5c7c5-qpzwv\" (UID: \"673eabe0-3a0e-4b7e-82f3-1653e4557d5c\") " pod="openshift-ingress/e2e-ait-f56a3efc-openshift-default-588cd5c7c5-qpzwv"
Oct 30 22:37:38.982355 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:38.982277    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/673eabe0-3a0e-4b7e-82f3-1653e4557d5c-istiod-ca-cert\") pod \"e2e-ait-f56a3efc-openshift-default-588cd5c7c5-qpzwv\" (UID: \"673eabe0-3a0e-4b7e-82f3-1653e4557d5c\") " pod="openshift-ingress/e2e-ait-f56a3efc-openshift-default-588cd5c7c5-qpzwv"
Oct 30 22:37:38.982632 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:38.982607    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/673eabe0-3a0e-4b7e-82f3-1653e4557d5c-workload-socket\") pod \"e2e-ait-f56a3efc-openshift-default-588cd5c7c5-qpzwv\" (UID: \"673eabe0-3a0e-4b7e-82f3-1653e4557d5c\") " pod="openshift-ingress/e2e-ait-f56a3efc-openshift-default-588cd5c7c5-qpzwv"
Oct 30 22:37:38.984291 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:38.984264    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/673eabe0-3a0e-4b7e-82f3-1653e4557d5c-istio-envoy\") pod \"e2e-ait-f56a3efc-openshift-default-588cd5c7c5-qpzwv\" (UID: \"673eabe0-3a0e-4b7e-82f3-1653e4557d5c\") " pod="openshift-ingress/e2e-ait-f56a3efc-openshift-default-588cd5c7c5-qpzwv"
Oct 30 22:37:38.984803 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:38.984781    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/673eabe0-3a0e-4b7e-82f3-1653e4557d5c-istio-podinfo\") pod \"e2e-ait-f56a3efc-openshift-default-588cd5c7c5-qpzwv\" (UID: \"673eabe0-3a0e-4b7e-82f3-1653e4557d5c\") " pod="openshift-ingress/e2e-ait-f56a3efc-openshift-default-588cd5c7c5-qpzwv"
Oct 30 22:37:39.039636 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:39.039605    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/673eabe0-3a0e-4b7e-82f3-1653e4557d5c-istio-token\") pod \"e2e-ait-f56a3efc-openshift-default-588cd5c7c5-qpzwv\" (UID: \"673eabe0-3a0e-4b7e-82f3-1653e4557d5c\") " pod="openshift-ingress/e2e-ait-f56a3efc-openshift-default-588cd5c7c5-qpzwv"
Oct 30 22:37:39.039817 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:39.039726    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pntsd\" (UniqueName: \"kubernetes.io/projected/673eabe0-3a0e-4b7e-82f3-1653e4557d5c-kube-api-access-pntsd\") pod \"e2e-ait-f56a3efc-openshift-default-588cd5c7c5-qpzwv\" (UID: \"673eabe0-3a0e-4b7e-82f3-1653e4557d5c\") " pod="openshift-ingress/e2e-ait-f56a3efc-openshift-default-588cd5c7c5-qpzwv"
Oct 30 22:37:39.186838 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:39.186737    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-f56a3efc-openshift-default-588cd5c7c5-qpzwv"
Oct 30 22:37:39.615259 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:39.615230    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-f56a3efc-openshift-default-588cd5c7c5-qpzwv"]
Oct 30 22:37:39.617150 ip-10-0-140-18 kubenswrapper[4473]: W1030 22:37:39.617122    4473 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod673eabe0_3a0e_4b7e_82f3_1653e4557d5c.slice/crio-a177c187f53a4a4020554ddbf9bc8becf30859c8cc9cbdb66791302ded5d32d7 WatchSource:0}: Error finding container a177c187f53a4a4020554ddbf9bc8becf30859c8cc9cbdb66791302ded5d32d7: Status 404 returned error can't find the container with id a177c187f53a4a4020554ddbf9bc8becf30859c8cc9cbdb66791302ded5d32d7
Oct 30 22:37:39.619425 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:39.619385    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:37:39.619536 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:39.619471    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:37:39.619536 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:39.619517    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:37:40.521180 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:40.521152    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-64979f7667-qkx6z"
Oct 30 22:37:40.540625 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:40.540593    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f56a3efc-openshift-default-588cd5c7c5-qpzwv" event={"ID":"673eabe0-3a0e-4b7e-82f3-1653e4557d5c","Type":"ContainerStarted","Data":"e2ae4fabb1e5817b1f14f6f8761f587fdf5e876344e6c48d6d6ca1eecd944071"}
Oct 30 22:37:40.540763 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:40.540647    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f56a3efc-openshift-default-588cd5c7c5-qpzwv" event={"ID":"673eabe0-3a0e-4b7e-82f3-1653e4557d5c","Type":"ContainerStarted","Data":"a177c187f53a4a4020554ddbf9bc8becf30859c8cc9cbdb66791302ded5d32d7"}
Oct 30 22:37:40.704132 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:40.704091    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-79787b59b7-d8gzw"]
Oct 30 22:37:40.704396 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:40.704345    4473 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-79787b59b7-d8gzw" podUID="c3ecd156-ac60-4605-86bb-e531fdcd1ce0" containerName="maas-api" containerID="cri-o://e6a569add08db5105c87008b0660d7985da223d3779609af911b0c7c1c650310" gracePeriod=30
Oct 30 22:37:40.806950 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:40.806817    4473 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-f56a3efc-openshift-default-588cd5c7c5-qpzwv" podStartSLOduration=2.806801525 podStartE2EDuration="2.806801525s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:37:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:37:40.796539321 +0000 UTC m=+3306.569907809" watchObservedRunningTime="2026-10-30 22:37:40.806801525 +0000 UTC m=+3306.580170012"
Oct 30 22:37:41.187291 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:41.187263    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-f56a3efc-openshift-default-588cd5c7c5-qpzwv"
Oct 30 22:37:41.192273 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:41.192250    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-f56a3efc-openshift-default-588cd5c7c5-qpzwv"
Oct 30 22:37:41.546227 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:41.546142    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-f56a3efc-openshift-default-588cd5c7c5-qpzwv"
Oct 30 22:37:41.547101 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:41.547082    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-f56a3efc-openshift-default-588cd5c7c5-qpzwv"
Oct 30 22:37:41.954360 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:41.954333    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-79787b59b7-d8gzw"
Oct 30 22:37:42.012450 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:42.012417    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c3ecd156-ac60-4605-86bb-e531fdcd1ce0-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c3ecd156-ac60-4605-86bb-e531fdcd1ce0-maas-api-tls\") pod \"c3ecd156-ac60-4605-86bb-e531fdcd1ce0\" (UID: \"c3ecd156-ac60-4605-86bb-e531fdcd1ce0\") "
Oct 30 22:37:42.012644 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:42.012551    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c3ecd156-ac60-4605-86bb-e531fdcd1ce0-metrics-certs\" (UniqueName: \"kubernetes.io/secret/c3ecd156-ac60-4605-86bb-e531fdcd1ce0-metrics-certs\") pod \"c3ecd156-ac60-4605-86bb-e531fdcd1ce0\" (UID: \"c3ecd156-ac60-4605-86bb-e531fdcd1ce0\") "
Oct 30 22:37:42.012644 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:42.012617    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c3ecd156-ac60-4605-86bb-e531fdcd1ce0-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c3ecd156-ac60-4605-86bb-e531fdcd1ce0-openshift-service-ca\") pod \"c3ecd156-ac60-4605-86bb-e531fdcd1ce0\" (UID: \"c3ecd156-ac60-4605-86bb-e531fdcd1ce0\") "
Oct 30 22:37:42.012644 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:42.012641    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c3ecd156-ac60-4605-86bb-e531fdcd1ce0-kube-api-access-glb95\" (UniqueName: \"kubernetes.io/projected/c3ecd156-ac60-4605-86bb-e531fdcd1ce0-kube-api-access-glb95\") pod \"c3ecd156-ac60-4605-86bb-e531fdcd1ce0\" (UID: \"c3ecd156-ac60-4605-86bb-e531fdcd1ce0\") "
Oct 30 22:37:42.013058 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:42.013026    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c3ecd156-ac60-4605-86bb-e531fdcd1ce0-openshift-service-ca" pod "c3ecd156-ac60-4605-86bb-e531fdcd1ce0" (UID: "c3ecd156-ac60-4605-86bb-e531fdcd1ce0"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:37:42.014790 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:42.014759    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c3ecd156-ac60-4605-86bb-e531fdcd1ce0-maas-api-tls" pod "c3ecd156-ac60-4605-86bb-e531fdcd1ce0" (UID: "c3ecd156-ac60-4605-86bb-e531fdcd1ce0"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:37:42.014882 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:42.014791    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c3ecd156-ac60-4605-86bb-e531fdcd1ce0-metrics-certs" pod "c3ecd156-ac60-4605-86bb-e531fdcd1ce0" (UID: "c3ecd156-ac60-4605-86bb-e531fdcd1ce0"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:37:42.015018 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:42.014888    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c3ecd156-ac60-4605-86bb-e531fdcd1ce0-kube-api-access-glb95" pod "c3ecd156-ac60-4605-86bb-e531fdcd1ce0" (UID: "c3ecd156-ac60-4605-86bb-e531fdcd1ce0"). InnerVolumeSpecName "kube-api-access-glb95". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:37:42.114074 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:42.114048    4473 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c3ecd156-ac60-4605-86bb-e531fdcd1ce0-maas-api-tls\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:37:42.114223 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:42.114077    4473 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c3ecd156-ac60-4605-86bb-e531fdcd1ce0-metrics-certs\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:37:42.114223 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:42.114087    4473 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c3ecd156-ac60-4605-86bb-e531fdcd1ce0-openshift-service-ca\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:37:42.114223 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:42.114096    4473 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-glb95\" (UniqueName: \"kubernetes.io/projected/c3ecd156-ac60-4605-86bb-e531fdcd1ce0-kube-api-access-glb95\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:37:42.551268 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:42.551187    4473 generic.go:356] "Generic (PLEG): container finished" podID="c3ecd156-ac60-4605-86bb-e531fdcd1ce0" containerID="e6a569add08db5105c87008b0660d7985da223d3779609af911b0c7c1c650310" exitCode=0
Oct 30 22:37:42.551268 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:42.551228    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-79787b59b7-d8gzw"
Oct 30 22:37:42.551779 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:42.551273    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-79787b59b7-d8gzw" event={"ID":"c3ecd156-ac60-4605-86bb-e531fdcd1ce0","Type":"ContainerDied","Data":"e6a569add08db5105c87008b0660d7985da223d3779609af911b0c7c1c650310"}
Oct 30 22:37:42.551779 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:42.551326    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-79787b59b7-d8gzw" event={"ID":"c3ecd156-ac60-4605-86bb-e531fdcd1ce0","Type":"ContainerDied","Data":"483d6d04c25e4b89a2619f49a69009cc4f14e20cf238767fd9e31e27b8cd85dd"}
Oct 30 22:37:42.551779 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:42.551352    4473 scope.go:122] "RemoveContainer" containerID="e6a569add08db5105c87008b0660d7985da223d3779609af911b0c7c1c650310"
Oct 30 22:37:42.560554 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:42.560535    4473 scope.go:122] "RemoveContainer" containerID="e6a569add08db5105c87008b0660d7985da223d3779609af911b0c7c1c650310"
Oct 30 22:37:42.560845 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:37:42.560823    4473 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e6a569add08db5105c87008b0660d7985da223d3779609af911b0c7c1c650310\": container with ID starting with e6a569add08db5105c87008b0660d7985da223d3779609af911b0c7c1c650310 not found: ID does not exist" containerID="e6a569add08db5105c87008b0660d7985da223d3779609af911b0c7c1c650310"
Oct 30 22:37:42.560909 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:42.560857    4473 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e6a569add08db5105c87008b0660d7985da223d3779609af911b0c7c1c650310"} err="failed to get container status \"e6a569add08db5105c87008b0660d7985da223d3779609af911b0c7c1c650310\": rpc error: code = NotFound desc = could not find container \"e6a569add08db5105c87008b0660d7985da223d3779609af911b0c7c1c650310\": container with ID starting with e6a569add08db5105c87008b0660d7985da223d3779609af911b0c7c1c650310 not found: ID does not exist"
Oct 30 22:37:42.716435 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:42.716400    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-79787b59b7-d8gzw"]
Oct 30 22:37:42.767003 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:42.766971    4473 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-79787b59b7-d8gzw"]
Oct 30 22:37:44.356177 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:44.356140    4473 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c3ecd156-ac60-4605-86bb-e531fdcd1ce0" path="/var/lib/kubelet/pods/c3ecd156-ac60-4605-86bb-e531fdcd1ce0/volumes"
Oct 30 22:37:46.601219 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:46.601184    4473 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-f56a3efc-65b4458675-ltsnw"]
Oct 30 22:37:46.601657 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:46.601625    4473 cpu_manager.go:414] "containerMap: removing container" podUID="c3ecd156-ac60-4605-86bb-e531fdcd1ce0" containerName="maas-api"
Oct 30 22:37:46.601657 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:46.601644    4473 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c3ecd156-ac60-4605-86bb-e531fdcd1ce0" containerName="maas-api"
Oct 30 22:37:46.601873 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:46.601784    4473 memory_manager.go:365] "RemoveStaleState removing state" podUID="c3ecd156-ac60-4605-86bb-e531fdcd1ce0" containerName="maas-api"
Oct 30 22:37:46.606356 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:46.606337    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-f56a3efc-65b4458675-ltsnw"
Oct 30 22:37:46.659001 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:46.658964    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g8ccm\" (UniqueName: \"kubernetes.io/projected/9e5ce512-6e69-40f2-a47b-b9e559489178-kube-api-access-g8ccm\") pod \"payload-pre-processing-e2e-ait-f56a3efc-65b4458675-ltsnw\" (UID: \"9e5ce512-6e69-40f2-a47b-b9e559489178\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-f56a3efc-65b4458675-ltsnw"
Oct 30 22:37:46.659391 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:46.659348    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9e5ce512-6e69-40f2-a47b-b9e559489178-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-f56a3efc-65b4458675-ltsnw\" (UID: \"9e5ce512-6e69-40f2-a47b-b9e559489178\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-f56a3efc-65b4458675-ltsnw"
Oct 30 22:37:46.663027 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:46.663000    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-f56a3efc-65b4458675-ltsnw"]
Oct 30 22:37:46.710256 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:46.710203    4473 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-f56a3efc\""
Oct 30 22:37:46.710467 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:46.710307    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-f56a3efc-dockercfg-bq9tj\""
Oct 30 22:37:46.760264 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:46.760220    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g8ccm\" (UniqueName: \"kubernetes.io/projected/9e5ce512-6e69-40f2-a47b-b9e559489178-kube-api-access-g8ccm\") pod \"payload-pre-processing-e2e-ait-f56a3efc-65b4458675-ltsnw\" (UID: \"9e5ce512-6e69-40f2-a47b-b9e559489178\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-f56a3efc-65b4458675-ltsnw"
Oct 30 22:37:46.760441 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:46.760276    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9e5ce512-6e69-40f2-a47b-b9e559489178-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-f56a3efc-65b4458675-ltsnw\" (UID: \"9e5ce512-6e69-40f2-a47b-b9e559489178\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-f56a3efc-65b4458675-ltsnw"
Oct 30 22:37:46.761042 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:46.761015    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9e5ce512-6e69-40f2-a47b-b9e559489178-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-f56a3efc-65b4458675-ltsnw\" (UID: \"9e5ce512-6e69-40f2-a47b-b9e559489178\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-f56a3efc-65b4458675-ltsnw"
Oct 30 22:37:46.762542 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:46.762522    4473 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-f56a3efc-58f7fbff8f-s79w2"]
Oct 30 22:37:46.766016 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:46.765997    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-f56a3efc-58f7fbff8f-s79w2"
Oct 30 22:37:46.812221 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:46.812183    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-f56a3efc-58f7fbff8f-s79w2"]
Oct 30 22:37:46.819577 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:46.819544    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g8ccm\" (UniqueName: \"kubernetes.io/projected/9e5ce512-6e69-40f2-a47b-b9e559489178-kube-api-access-g8ccm\") pod \"payload-pre-processing-e2e-ait-f56a3efc-65b4458675-ltsnw\" (UID: \"9e5ce512-6e69-40f2-a47b-b9e559489178\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-f56a3efc-65b4458675-ltsnw"
Oct 30 22:37:46.861070 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:46.860986    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4m5tr\" (UniqueName: \"kubernetes.io/projected/0baff427-0e69-4480-aaf0-efc881e33fed-kube-api-access-4m5tr\") pod \"payload-processing-e2e-ait-f56a3efc-58f7fbff8f-s79w2\" (UID: \"0baff427-0e69-4480-aaf0-efc881e33fed\") " pod="openshift-ingress/payload-processing-e2e-ait-f56a3efc-58f7fbff8f-s79w2"
Oct 30 22:37:46.861235 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:46.861123    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0baff427-0e69-4480-aaf0-efc881e33fed-plugins-config-volume\") pod \"payload-processing-e2e-ait-f56a3efc-58f7fbff8f-s79w2\" (UID: \"0baff427-0e69-4480-aaf0-efc881e33fed\") " pod="openshift-ingress/payload-processing-e2e-ait-f56a3efc-58f7fbff8f-s79w2"
Oct 30 22:37:46.920228 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:46.920184    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-f56a3efc-65b4458675-ltsnw"
Oct 30 22:37:46.962024 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:46.961992    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4m5tr\" (UniqueName: \"kubernetes.io/projected/0baff427-0e69-4480-aaf0-efc881e33fed-kube-api-access-4m5tr\") pod \"payload-processing-e2e-ait-f56a3efc-58f7fbff8f-s79w2\" (UID: \"0baff427-0e69-4480-aaf0-efc881e33fed\") " pod="openshift-ingress/payload-processing-e2e-ait-f56a3efc-58f7fbff8f-s79w2"
Oct 30 22:37:46.962215 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:46.962106    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0baff427-0e69-4480-aaf0-efc881e33fed-plugins-config-volume\") pod \"payload-processing-e2e-ait-f56a3efc-58f7fbff8f-s79w2\" (UID: \"0baff427-0e69-4480-aaf0-efc881e33fed\") " pod="openshift-ingress/payload-processing-e2e-ait-f56a3efc-58f7fbff8f-s79w2"
Oct 30 22:37:46.964377 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:46.963035    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0baff427-0e69-4480-aaf0-efc881e33fed-plugins-config-volume\") pod \"payload-processing-e2e-ait-f56a3efc-58f7fbff8f-s79w2\" (UID: \"0baff427-0e69-4480-aaf0-efc881e33fed\") " pod="openshift-ingress/payload-processing-e2e-ait-f56a3efc-58f7fbff8f-s79w2"
Oct 30 22:37:47.019913 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:47.019884    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4m5tr\" (UniqueName: \"kubernetes.io/projected/0baff427-0e69-4480-aaf0-efc881e33fed-kube-api-access-4m5tr\") pod \"payload-processing-e2e-ait-f56a3efc-58f7fbff8f-s79w2\" (UID: \"0baff427-0e69-4480-aaf0-efc881e33fed\") " pod="openshift-ingress/payload-processing-e2e-ait-f56a3efc-58f7fbff8f-s79w2"
Oct 30 22:37:47.045760 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:47.045728    4473 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f56a3efc-794d6757cd-mnjm2"]
Oct 30 22:37:47.050893 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:47.050868    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f56a3efc-794d6757cd-mnjm2"
Oct 30 22:37:47.078953 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:47.078905    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-f56a3efc-58f7fbff8f-s79w2"
Oct 30 22:37:47.106012 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:47.105979    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-f56a3efc-serving-cert\""
Oct 30 22:37:47.109270 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:47.109232    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f56a3efc-794d6757cd-mnjm2"]
Oct 30 22:37:47.164712 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:47.164670    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/91ff8841-1df2-434d-b422-fbf79c40b844-metrics-certs\") pod \"maas-api-e2e-ait-f56a3efc-794d6757cd-mnjm2\" (UID: \"91ff8841-1df2-434d-b422-fbf79c40b844\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f56a3efc-794d6757cd-mnjm2"
Oct 30 22:37:47.164862 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:47.164801    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/91ff8841-1df2-434d-b422-fbf79c40b844-maas-api-tls\") pod \"maas-api-e2e-ait-f56a3efc-794d6757cd-mnjm2\" (UID: \"91ff8841-1df2-434d-b422-fbf79c40b844\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f56a3efc-794d6757cd-mnjm2"
Oct 30 22:37:47.164862 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:47.164843    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-82dmp\" (UniqueName: \"kubernetes.io/projected/91ff8841-1df2-434d-b422-fbf79c40b844-kube-api-access-82dmp\") pod \"maas-api-e2e-ait-f56a3efc-794d6757cd-mnjm2\" (UID: \"91ff8841-1df2-434d-b422-fbf79c40b844\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f56a3efc-794d6757cd-mnjm2"
Oct 30 22:37:47.164985 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:47.164905    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/91ff8841-1df2-434d-b422-fbf79c40b844-openshift-service-ca\") pod \"maas-api-e2e-ait-f56a3efc-794d6757cd-mnjm2\" (UID: \"91ff8841-1df2-434d-b422-fbf79c40b844\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f56a3efc-794d6757cd-mnjm2"
Oct 30 22:37:47.266155 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:47.266116    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/91ff8841-1df2-434d-b422-fbf79c40b844-metrics-certs\") pod \"maas-api-e2e-ait-f56a3efc-794d6757cd-mnjm2\" (UID: \"91ff8841-1df2-434d-b422-fbf79c40b844\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f56a3efc-794d6757cd-mnjm2"
Oct 30 22:37:47.266343 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:47.266201    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/91ff8841-1df2-434d-b422-fbf79c40b844-maas-api-tls\") pod \"maas-api-e2e-ait-f56a3efc-794d6757cd-mnjm2\" (UID: \"91ff8841-1df2-434d-b422-fbf79c40b844\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f56a3efc-794d6757cd-mnjm2"
Oct 30 22:37:47.266343 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:47.266226    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-82dmp\" (UniqueName: \"kubernetes.io/projected/91ff8841-1df2-434d-b422-fbf79c40b844-kube-api-access-82dmp\") pod \"maas-api-e2e-ait-f56a3efc-794d6757cd-mnjm2\" (UID: \"91ff8841-1df2-434d-b422-fbf79c40b844\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f56a3efc-794d6757cd-mnjm2"
Oct 30 22:37:47.266343 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:47.266261    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/91ff8841-1df2-434d-b422-fbf79c40b844-openshift-service-ca\") pod \"maas-api-e2e-ait-f56a3efc-794d6757cd-mnjm2\" (UID: \"91ff8841-1df2-434d-b422-fbf79c40b844\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f56a3efc-794d6757cd-mnjm2"
Oct 30 22:37:47.266447 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:37:47.266351    4473 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-f56a3efc-serving-cert: secret "maas-api-e2e-ait-f56a3efc-serving-cert" not found
Oct 30 22:37:47.266447 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:37:47.266424    4473 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/91ff8841-1df2-434d-b422-fbf79c40b844-maas-api-tls podName:91ff8841-1df2-434d-b422-fbf79c40b844 nodeName:}" failed. No retries permitted until 2026-10-30 22:37:47.766404021 +0000 UTC m=+3313.539772487 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/91ff8841-1df2-434d-b422-fbf79c40b844-maas-api-tls") pod "maas-api-e2e-ait-f56a3efc-794d6757cd-mnjm2" (UID: "91ff8841-1df2-434d-b422-fbf79c40b844") : secret "maas-api-e2e-ait-f56a3efc-serving-cert" not found
Oct 30 22:37:47.266909 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:47.266885    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/91ff8841-1df2-434d-b422-fbf79c40b844-openshift-service-ca\") pod \"maas-api-e2e-ait-f56a3efc-794d6757cd-mnjm2\" (UID: \"91ff8841-1df2-434d-b422-fbf79c40b844\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f56a3efc-794d6757cd-mnjm2"
Oct 30 22:37:47.268881 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:47.268857    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/91ff8841-1df2-434d-b422-fbf79c40b844-metrics-certs\") pod \"maas-api-e2e-ait-f56a3efc-794d6757cd-mnjm2\" (UID: \"91ff8841-1df2-434d-b422-fbf79c40b844\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f56a3efc-794d6757cd-mnjm2"
Oct 30 22:37:47.297264 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:47.297240    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-f56a3efc-65b4458675-ltsnw"]
Oct 30 22:37:47.300074 ip-10-0-140-18 kubenswrapper[4473]: W1030 22:37:47.299279    4473 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e5ce512_6e69_40f2_a47b_b9e559489178.slice/crio-2c554be8430bae073d684d112b432d6787265e0accf6c64a8592fcf54201e235 WatchSource:0}: Error finding container 2c554be8430bae073d684d112b432d6787265e0accf6c64a8592fcf54201e235: Status 404 returned error can't find the container with id 2c554be8430bae073d684d112b432d6787265e0accf6c64a8592fcf54201e235
Oct 30 22:37:47.325028 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:47.325000    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-82dmp\" (UniqueName: \"kubernetes.io/projected/91ff8841-1df2-434d-b422-fbf79c40b844-kube-api-access-82dmp\") pod \"maas-api-e2e-ait-f56a3efc-794d6757cd-mnjm2\" (UID: \"91ff8841-1df2-434d-b422-fbf79c40b844\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f56a3efc-794d6757cd-mnjm2"
Oct 30 22:37:47.461563 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:47.461173    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-f56a3efc-58f7fbff8f-s79w2"]
Oct 30 22:37:47.577349 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:47.577306    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-f56a3efc-65b4458675-ltsnw" event={"ID":"9e5ce512-6e69-40f2-a47b-b9e559489178","Type":"ContainerStarted","Data":"76121780c11dec78631390b4caf470588b319ea18fa288929adf46b15e963614"}
Oct 30 22:37:47.577550 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:47.577378    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-f56a3efc-65b4458675-ltsnw" event={"ID":"9e5ce512-6e69-40f2-a47b-b9e559489178","Type":"ContainerStarted","Data":"2c554be8430bae073d684d112b432d6787265e0accf6c64a8592fcf54201e235"}
Oct 30 22:37:47.577550 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:47.577410    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-f56a3efc-65b4458675-ltsnw"
Oct 30 22:37:47.578959 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:47.578907    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-f56a3efc-58f7fbff8f-s79w2" event={"ID":"0baff427-0e69-4480-aaf0-efc881e33fed","Type":"ContainerStarted","Data":"f6e8cb738842f14e274ba307f8baee4f2750571ef4f9a774bf3dc3b0102733dd"}
Oct 30 22:37:47.579092 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:47.578974    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-f56a3efc-58f7fbff8f-s79w2" event={"ID":"0baff427-0e69-4480-aaf0-efc881e33fed","Type":"ContainerStarted","Data":"e040cbdd43b2c984d3c1905124ecfa3052e2705edee1c01e47a6e0bffa3318f9"}
Oct 30 22:37:47.579092 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:47.578996    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-f56a3efc-58f7fbff8f-s79w2"
Oct 30 22:37:47.692444 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:47.692385    4473 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-f56a3efc-65b4458675-ltsnw" podStartSLOduration=1.69236641 podStartE2EDuration="1.69236641s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:37:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:37:47.691404745 +0000 UTC m=+3313.464773232" watchObservedRunningTime="2026-10-30 22:37:47.69236641 +0000 UTC m=+3313.465734898"
Oct 30 22:37:47.772997 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:47.772872    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/91ff8841-1df2-434d-b422-fbf79c40b844-maas-api-tls\") pod \"maas-api-e2e-ait-f56a3efc-794d6757cd-mnjm2\" (UID: \"91ff8841-1df2-434d-b422-fbf79c40b844\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f56a3efc-794d6757cd-mnjm2"
Oct 30 22:37:47.775556 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:47.775525    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/91ff8841-1df2-434d-b422-fbf79c40b844-maas-api-tls\") pod \"maas-api-e2e-ait-f56a3efc-794d6757cd-mnjm2\" (UID: \"91ff8841-1df2-434d-b422-fbf79c40b844\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f56a3efc-794d6757cd-mnjm2"
Oct 30 22:37:47.809731 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:47.809658    4473 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-f56a3efc-58f7fbff8f-s79w2" podStartSLOduration=1.809641449 podStartE2EDuration="1.809641449s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:37:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:37:47.805745937 +0000 UTC m=+3313.579114425" watchObservedRunningTime="2026-10-30 22:37:47.809641449 +0000 UTC m=+3313.583009935"
Oct 30 22:37:47.965582 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:47.965552    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f56a3efc-794d6757cd-mnjm2"
Oct 30 22:37:48.391632 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:48.391601    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f56a3efc-794d6757cd-mnjm2"]
Oct 30 22:37:48.393200 ip-10-0-140-18 kubenswrapper[4473]: W1030 22:37:48.393172    4473 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91ff8841_1df2_434d_b422_fbf79c40b844.slice/crio-19bb0c204b155865387d8d625576bfb1c84c704da9ec0bd3fe87046488d79f6b WatchSource:0}: Error finding container 19bb0c204b155865387d8d625576bfb1c84c704da9ec0bd3fe87046488d79f6b: Status 404 returned error can't find the container with id 19bb0c204b155865387d8d625576bfb1c84c704da9ec0bd3fe87046488d79f6b
Oct 30 22:37:48.585311 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:48.585273    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f56a3efc-794d6757cd-mnjm2" event={"ID":"91ff8841-1df2-434d-b422-fbf79c40b844","Type":"ContainerStarted","Data":"19bb0c204b155865387d8d625576bfb1c84c704da9ec0bd3fe87046488d79f6b"}
Oct 30 22:37:49.592415 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:49.592326    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f56a3efc-794d6757cd-mnjm2" event={"ID":"91ff8841-1df2-434d-b422-fbf79c40b844","Type":"ContainerStarted","Data":"fd0bf253af4a07cb1bdfd91a9671f9ed588648726956632a00d0f95ba56884ba"}
Oct 30 22:37:49.592816 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:49.592463    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f56a3efc-794d6757cd-mnjm2"
Oct 30 22:37:49.708930 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:49.708874    4473 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f56a3efc-794d6757cd-mnjm2" podStartSLOduration=1.8097270939999999 podStartE2EDuration="2.708859403s" totalImagesPullingTime="899.132309ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:37:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:37:49.708310863 +0000 UTC m=+3315.481679349" watchObservedRunningTime="2026-10-30 22:37:49.708859403 +0000 UTC m=+3315.482227889"
Oct 30 22:37:50.324185 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:37:50.324142    4473 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fncvg" containerName="extractor"
Oct 30 22:37:55.603743 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:55.603716    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f56a3efc-794d6757cd-mnjm2"
Oct 30 22:37:58.589028 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:58.588997    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-f56a3efc-58f7fbff8f-s79w2"
Oct 30 22:37:58.589462 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:37:58.589064    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-f56a3efc-65b4458675-ltsnw"
Oct 30 22:38:04.696619 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:04.696531    4473 generic.go:356] "Generic (PLEG): container finished" podID="673eabe0-3a0e-4b7e-82f3-1653e4557d5c" containerID="e2ae4fabb1e5817b1f14f6f8761f587fdf5e876344e6c48d6d6ca1eecd944071" exitCode=0
Oct 30 22:38:04.697073 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:04.696609    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f56a3efc-openshift-default-588cd5c7c5-qpzwv" event={"ID":"673eabe0-3a0e-4b7e-82f3-1653e4557d5c","Type":"ContainerDied","Data":"e2ae4fabb1e5817b1f14f6f8761f587fdf5e876344e6c48d6d6ca1eecd944071"}
Oct 30 22:38:04.697073 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:04.697021    4473 scope.go:122] "RemoveContainer" containerID="e2ae4fabb1e5817b1f14f6f8761f587fdf5e876344e6c48d6d6ca1eecd944071"
Oct 30 22:38:04.698241 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:04.698211    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:38:04.698369 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:04.698296    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:38:04.698369 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:04.698346    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:38:05.704375 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:05.704344    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f56a3efc-openshift-default-588cd5c7c5-qpzwv" event={"ID":"673eabe0-3a0e-4b7e-82f3-1653e4557d5c","Type":"ContainerStarted","Data":"50f92f0dade44df03903db95007bed03ef2f9f25401c6588803ae35aedc2b11d"}
Oct 30 22:38:06.187826 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:06.187788    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-f56a3efc-openshift-default-588cd5c7c5-qpzwv"
Oct 30 22:38:06.188137 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:06.188116    4473 patch_prober.go:29] interesting pod/e2e-ait-f56a3efc-openshift-default-588cd5c7c5-qpzwv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.163:15021/healthz/ready\": dial tcp 10.133.0.163:15021: connect: connection refused" start-of-body=
Oct 30 22:38:06.188194 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:06.188171    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-f56a3efc-openshift-default-588cd5c7c5-qpzwv" podUID="673eabe0-3a0e-4b7e-82f3-1653e4557d5c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.163:15021/healthz/ready\": dial tcp 10.133.0.163:15021: connect: connection refused"
Oct 30 22:38:07.187422 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:07.187386    4473 patch_prober.go:29] interesting pod/e2e-ait-f56a3efc-openshift-default-588cd5c7c5-qpzwv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.163:15021/healthz/ready\": dial tcp 10.133.0.163:15021: connect: connection refused" start-of-body=
Oct 30 22:38:07.187853 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:07.187450    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-f56a3efc-openshift-default-588cd5c7c5-qpzwv" podUID="673eabe0-3a0e-4b7e-82f3-1653e4557d5c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.163:15021/healthz/ready\": dial tcp 10.133.0.163:15021: connect: connection refused"
Oct 30 22:38:08.187522 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:08.187487    4473 patch_prober.go:29] interesting pod/e2e-ait-f56a3efc-openshift-default-588cd5c7c5-qpzwv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.163:15021/healthz/ready\": dial tcp 10.133.0.163:15021: connect: connection refused" start-of-body=
Oct 30 22:38:08.188005 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:08.187541    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-f56a3efc-openshift-default-588cd5c7c5-qpzwv" podUID="673eabe0-3a0e-4b7e-82f3-1653e4557d5c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.163:15021/healthz/ready\": dial tcp 10.133.0.163:15021: connect: connection refused"
Oct 30 22:38:09.187205 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:09.187171    4473 patch_prober.go:29] interesting pod/e2e-ait-f56a3efc-openshift-default-588cd5c7c5-qpzwv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.163:15021/healthz/ready\": dial tcp 10.133.0.163:15021: connect: connection refused" start-of-body=
Oct 30 22:38:09.187423 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:09.187232    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-f56a3efc-openshift-default-588cd5c7c5-qpzwv" podUID="673eabe0-3a0e-4b7e-82f3-1653e4557d5c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.163:15021/healthz/ready\": dial tcp 10.133.0.163:15021: connect: connection refused"
Oct 30 22:38:10.187310 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:10.187268    4473 patch_prober.go:29] interesting pod/e2e-ait-f56a3efc-openshift-default-588cd5c7c5-qpzwv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.163:15021/healthz/ready\": dial tcp 10.133.0.163:15021: connect: connection refused" start-of-body=
Oct 30 22:38:10.187754 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:10.187344    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-f56a3efc-openshift-default-588cd5c7c5-qpzwv" podUID="673eabe0-3a0e-4b7e-82f3-1653e4557d5c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.163:15021/healthz/ready\": dial tcp 10.133.0.163:15021: connect: connection refused"
Oct 30 22:38:11.191651 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:11.191620    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-f56a3efc-openshift-default-588cd5c7c5-qpzwv"
Oct 30 22:38:11.192118 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:11.192043    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-f56a3efc-openshift-default-588cd5c7c5-qpzwv"
Oct 30 22:38:11.192676 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:11.192661    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-f56a3efc-openshift-default-588cd5c7c5-qpzwv"
Oct 30 22:38:20.322972 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:38:20.322935    4473 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fncvg" containerName="extractor"
Oct 30 22:38:30.830471 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:30.830439    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-f56a3efc-58f7fbff8f-s79w2"]
Oct 30 22:38:30.935909 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:30.935875    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-f56a3efc-65b4458675-ltsnw"]
Oct 30 22:38:35.832983 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:35.832935    4473 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-f56a3efc-58f7fbff8f-s79w2" podUID="0baff427-0e69-4480-aaf0-efc881e33fed" containerName="payload-processing" containerID="cri-o://f6e8cb738842f14e274ba307f8baee4f2750571ef4f9a774bf3dc3b0102733dd" gracePeriod=25
Oct 30 22:38:35.936825 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:35.936743    4473 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-f56a3efc-65b4458675-ltsnw" podUID="9e5ce512-6e69-40f2-a47b-b9e559489178" containerName="payload-pre-processing" containerID="cri-o://76121780c11dec78631390b4caf470588b319ea18fa288929adf46b15e963614" gracePeriod=25
Oct 30 22:38:36.090075 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:36.090001    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-f56a3efc-58f7fbff8f-s79w2"
Oct 30 22:38:36.184560 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:36.184522    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0baff427-0e69-4480-aaf0-efc881e33fed-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0baff427-0e69-4480-aaf0-efc881e33fed-plugins-config-volume\") pod \"0baff427-0e69-4480-aaf0-efc881e33fed\" (UID: \"0baff427-0e69-4480-aaf0-efc881e33fed\") "
Oct 30 22:38:36.184757 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:36.184676    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0baff427-0e69-4480-aaf0-efc881e33fed-kube-api-access-4m5tr\" (UniqueName: \"kubernetes.io/projected/0baff427-0e69-4480-aaf0-efc881e33fed-kube-api-access-4m5tr\") pod \"0baff427-0e69-4480-aaf0-efc881e33fed\" (UID: \"0baff427-0e69-4480-aaf0-efc881e33fed\") "
Oct 30 22:38:36.184909 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:36.184884    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0baff427-0e69-4480-aaf0-efc881e33fed-plugins-config-volume" pod "0baff427-0e69-4480-aaf0-efc881e33fed" (UID: "0baff427-0e69-4480-aaf0-efc881e33fed"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:38:36.186997 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:36.186973    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0baff427-0e69-4480-aaf0-efc881e33fed-kube-api-access-4m5tr" pod "0baff427-0e69-4480-aaf0-efc881e33fed" (UID: "0baff427-0e69-4480-aaf0-efc881e33fed"). InnerVolumeSpecName "kube-api-access-4m5tr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:38:36.206282 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:36.206255    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-f56a3efc-65b4458675-ltsnw"
Oct 30 22:38:36.286301 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:36.286258    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9e5ce512-6e69-40f2-a47b-b9e559489178-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9e5ce512-6e69-40f2-a47b-b9e559489178-plugins-config-volume\") pod \"9e5ce512-6e69-40f2-a47b-b9e559489178\" (UID: \"9e5ce512-6e69-40f2-a47b-b9e559489178\") "
Oct 30 22:38:36.286454 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:36.286332    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9e5ce512-6e69-40f2-a47b-b9e559489178-kube-api-access-g8ccm\" (UniqueName: \"kubernetes.io/projected/9e5ce512-6e69-40f2-a47b-b9e559489178-kube-api-access-g8ccm\") pod \"9e5ce512-6e69-40f2-a47b-b9e559489178\" (UID: \"9e5ce512-6e69-40f2-a47b-b9e559489178\") "
Oct 30 22:38:36.286586 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:36.286570    4473 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0baff427-0e69-4480-aaf0-efc881e33fed-plugins-config-volume\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:38:36.286638 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:36.286591    4473 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4m5tr\" (UniqueName: \"kubernetes.io/projected/0baff427-0e69-4480-aaf0-efc881e33fed-kube-api-access-4m5tr\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:38:36.286771 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:36.286736    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9e5ce512-6e69-40f2-a47b-b9e559489178-plugins-config-volume" pod "9e5ce512-6e69-40f2-a47b-b9e559489178" (UID: "9e5ce512-6e69-40f2-a47b-b9e559489178"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:38:36.288673 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:36.288651    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9e5ce512-6e69-40f2-a47b-b9e559489178-kube-api-access-g8ccm" pod "9e5ce512-6e69-40f2-a47b-b9e559489178" (UID: "9e5ce512-6e69-40f2-a47b-b9e559489178"). InnerVolumeSpecName "kube-api-access-g8ccm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:38:36.357241 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:36.357159    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f56a3efc-794d6757cd-mnjm2"]
Oct 30 22:38:36.357403 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:36.357379    4473 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f56a3efc-794d6757cd-mnjm2" podUID="91ff8841-1df2-434d-b422-fbf79c40b844" containerName="maas-api" containerID="cri-o://fd0bf253af4a07cb1bdfd91a9671f9ed588648726956632a00d0f95ba56884ba" gracePeriod=30
Oct 30 22:38:36.388424 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:36.388384    4473 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9e5ce512-6e69-40f2-a47b-b9e559489178-plugins-config-volume\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:38:36.388424 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:36.388424    4473 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g8ccm\" (UniqueName: \"kubernetes.io/projected/9e5ce512-6e69-40f2-a47b-b9e559489178-kube-api-access-g8ccm\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:38:36.595162 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:36.595126    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f56a3efc-794d6757cd-mnjm2"
Oct 30 22:38:36.691163 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:36.691132    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/91ff8841-1df2-434d-b422-fbf79c40b844-kube-api-access-82dmp\" (UniqueName: \"kubernetes.io/projected/91ff8841-1df2-434d-b422-fbf79c40b844-kube-api-access-82dmp\") pod \"91ff8841-1df2-434d-b422-fbf79c40b844\" (UID: \"91ff8841-1df2-434d-b422-fbf79c40b844\") "
Oct 30 22:38:36.691356 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:36.691226    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/91ff8841-1df2-434d-b422-fbf79c40b844-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/91ff8841-1df2-434d-b422-fbf79c40b844-maas-api-tls\") pod \"91ff8841-1df2-434d-b422-fbf79c40b844\" (UID: \"91ff8841-1df2-434d-b422-fbf79c40b844\") "
Oct 30 22:38:36.691356 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:36.691322    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/91ff8841-1df2-434d-b422-fbf79c40b844-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/91ff8841-1df2-434d-b422-fbf79c40b844-openshift-service-ca\") pod \"91ff8841-1df2-434d-b422-fbf79c40b844\" (UID: \"91ff8841-1df2-434d-b422-fbf79c40b844\") "
Oct 30 22:38:36.691356 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:36.691350    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/91ff8841-1df2-434d-b422-fbf79c40b844-metrics-certs\" (UniqueName: \"kubernetes.io/secret/91ff8841-1df2-434d-b422-fbf79c40b844-metrics-certs\") pod \"91ff8841-1df2-434d-b422-fbf79c40b844\" (UID: \"91ff8841-1df2-434d-b422-fbf79c40b844\") "
Oct 30 22:38:36.691789 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:36.691756    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/91ff8841-1df2-434d-b422-fbf79c40b844-openshift-service-ca" pod "91ff8841-1df2-434d-b422-fbf79c40b844" (UID: "91ff8841-1df2-434d-b422-fbf79c40b844"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:38:36.693635 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:36.693600    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/91ff8841-1df2-434d-b422-fbf79c40b844-metrics-certs" pod "91ff8841-1df2-434d-b422-fbf79c40b844" (UID: "91ff8841-1df2-434d-b422-fbf79c40b844"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:38:36.693635 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:36.693624    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/91ff8841-1df2-434d-b422-fbf79c40b844-kube-api-access-82dmp" pod "91ff8841-1df2-434d-b422-fbf79c40b844" (UID: "91ff8841-1df2-434d-b422-fbf79c40b844"). InnerVolumeSpecName "kube-api-access-82dmp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:38:36.693635 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:36.693631    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/91ff8841-1df2-434d-b422-fbf79c40b844-maas-api-tls" pod "91ff8841-1df2-434d-b422-fbf79c40b844" (UID: "91ff8841-1df2-434d-b422-fbf79c40b844"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:38:36.792426 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:36.792391    4473 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/91ff8841-1df2-434d-b422-fbf79c40b844-openshift-service-ca\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:38:36.792426 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:36.792423    4473 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/91ff8841-1df2-434d-b422-fbf79c40b844-metrics-certs\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:38:36.792426 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:36.792432    4473 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-82dmp\" (UniqueName: \"kubernetes.io/projected/91ff8841-1df2-434d-b422-fbf79c40b844-kube-api-access-82dmp\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:38:36.792680 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:36.792442    4473 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/91ff8841-1df2-434d-b422-fbf79c40b844-maas-api-tls\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:38:36.876581 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:36.876550    4473 generic.go:356] "Generic (PLEG): container finished" podID="9e5ce512-6e69-40f2-a47b-b9e559489178" containerID="76121780c11dec78631390b4caf470588b319ea18fa288929adf46b15e963614" exitCode=0
Oct 30 22:38:36.877108 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:36.876608    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-f56a3efc-65b4458675-ltsnw"
Oct 30 22:38:36.877108 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:36.876639    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-f56a3efc-65b4458675-ltsnw" event={"ID":"9e5ce512-6e69-40f2-a47b-b9e559489178","Type":"ContainerDied","Data":"76121780c11dec78631390b4caf470588b319ea18fa288929adf46b15e963614"}
Oct 30 22:38:36.877108 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:36.876861    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-f56a3efc-65b4458675-ltsnw" event={"ID":"9e5ce512-6e69-40f2-a47b-b9e559489178","Type":"ContainerDied","Data":"2c554be8430bae073d684d112b432d6787265e0accf6c64a8592fcf54201e235"}
Oct 30 22:38:36.877108 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:36.876906    4473 scope.go:122] "RemoveContainer" containerID="76121780c11dec78631390b4caf470588b319ea18fa288929adf46b15e963614"
Oct 30 22:38:36.879393 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:36.879371    4473 generic.go:356] "Generic (PLEG): container finished" podID="0baff427-0e69-4480-aaf0-efc881e33fed" containerID="f6e8cb738842f14e274ba307f8baee4f2750571ef4f9a774bf3dc3b0102733dd" exitCode=0
Oct 30 22:38:36.879525 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:36.879442    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-f56a3efc-58f7fbff8f-s79w2"
Oct 30 22:38:36.879525 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:36.879505    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-f56a3efc-58f7fbff8f-s79w2" event={"ID":"0baff427-0e69-4480-aaf0-efc881e33fed","Type":"ContainerDied","Data":"f6e8cb738842f14e274ba307f8baee4f2750571ef4f9a774bf3dc3b0102733dd"}
Oct 30 22:38:36.879657 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:36.879550    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-f56a3efc-58f7fbff8f-s79w2" event={"ID":"0baff427-0e69-4480-aaf0-efc881e33fed","Type":"ContainerDied","Data":"e040cbdd43b2c984d3c1905124ecfa3052e2705edee1c01e47a6e0bffa3318f9"}
Oct 30 22:38:36.887959 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:36.882154    4473 generic.go:356] "Generic (PLEG): container finished" podID="91ff8841-1df2-434d-b422-fbf79c40b844" containerID="fd0bf253af4a07cb1bdfd91a9671f9ed588648726956632a00d0f95ba56884ba" exitCode=0
Oct 30 22:38:36.887959 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:36.882182    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f56a3efc-794d6757cd-mnjm2"
Oct 30 22:38:36.887959 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:36.882218    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f56a3efc-794d6757cd-mnjm2" event={"ID":"91ff8841-1df2-434d-b422-fbf79c40b844","Type":"ContainerDied","Data":"fd0bf253af4a07cb1bdfd91a9671f9ed588648726956632a00d0f95ba56884ba"}
Oct 30 22:38:36.887959 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:36.882256    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f56a3efc-794d6757cd-mnjm2" event={"ID":"91ff8841-1df2-434d-b422-fbf79c40b844","Type":"ContainerDied","Data":"19bb0c204b155865387d8d625576bfb1c84c704da9ec0bd3fe87046488d79f6b"}
Oct 30 22:38:36.889489 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:36.889462    4473 scope.go:122] "RemoveContainer" containerID="76121780c11dec78631390b4caf470588b319ea18fa288929adf46b15e963614"
Oct 30 22:38:36.889846 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:38:36.889824    4473 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"76121780c11dec78631390b4caf470588b319ea18fa288929adf46b15e963614\": container with ID starting with 76121780c11dec78631390b4caf470588b319ea18fa288929adf46b15e963614 not found: ID does not exist" containerID="76121780c11dec78631390b4caf470588b319ea18fa288929adf46b15e963614"
Oct 30 22:38:36.889996 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:36.889858    4473 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"76121780c11dec78631390b4caf470588b319ea18fa288929adf46b15e963614"} err="failed to get container status \"76121780c11dec78631390b4caf470588b319ea18fa288929adf46b15e963614\": rpc error: code = NotFound desc = could not find container \"76121780c11dec78631390b4caf470588b319ea18fa288929adf46b15e963614\": container with ID starting with 76121780c11dec78631390b4caf470588b319ea18fa288929adf46b15e963614 not found: ID does not exist"
Oct 30 22:38:36.889996 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:36.889880    4473 scope.go:122] "RemoveContainer" containerID="f6e8cb738842f14e274ba307f8baee4f2750571ef4f9a774bf3dc3b0102733dd"
Oct 30 22:38:36.898908 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:36.898869    4473 scope.go:122] "RemoveContainer" containerID="f6e8cb738842f14e274ba307f8baee4f2750571ef4f9a774bf3dc3b0102733dd"
Oct 30 22:38:36.899263 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:38:36.899241    4473 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f6e8cb738842f14e274ba307f8baee4f2750571ef4f9a774bf3dc3b0102733dd\": container with ID starting with f6e8cb738842f14e274ba307f8baee4f2750571ef4f9a774bf3dc3b0102733dd not found: ID does not exist" containerID="f6e8cb738842f14e274ba307f8baee4f2750571ef4f9a774bf3dc3b0102733dd"
Oct 30 22:38:36.899355 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:36.899276    4473 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f6e8cb738842f14e274ba307f8baee4f2750571ef4f9a774bf3dc3b0102733dd"} err="failed to get container status \"f6e8cb738842f14e274ba307f8baee4f2750571ef4f9a774bf3dc3b0102733dd\": rpc error: code = NotFound desc = could not find container \"f6e8cb738842f14e274ba307f8baee4f2750571ef4f9a774bf3dc3b0102733dd\": container with ID starting with f6e8cb738842f14e274ba307f8baee4f2750571ef4f9a774bf3dc3b0102733dd not found: ID does not exist"
Oct 30 22:38:36.899355 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:36.899301    4473 scope.go:122] "RemoveContainer" containerID="fd0bf253af4a07cb1bdfd91a9671f9ed588648726956632a00d0f95ba56884ba"
Oct 30 22:38:36.907708 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:36.907681    4473 scope.go:122] "RemoveContainer" containerID="fd0bf253af4a07cb1bdfd91a9671f9ed588648726956632a00d0f95ba56884ba"
Oct 30 22:38:36.908024 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:38:36.908004    4473 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fd0bf253af4a07cb1bdfd91a9671f9ed588648726956632a00d0f95ba56884ba\": container with ID starting with fd0bf253af4a07cb1bdfd91a9671f9ed588648726956632a00d0f95ba56884ba not found: ID does not exist" containerID="fd0bf253af4a07cb1bdfd91a9671f9ed588648726956632a00d0f95ba56884ba"
Oct 30 22:38:36.908109 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:36.908038    4473 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fd0bf253af4a07cb1bdfd91a9671f9ed588648726956632a00d0f95ba56884ba"} err="failed to get container status \"fd0bf253af4a07cb1bdfd91a9671f9ed588648726956632a00d0f95ba56884ba\": rpc error: code = NotFound desc = could not find container \"fd0bf253af4a07cb1bdfd91a9671f9ed588648726956632a00d0f95ba56884ba\": container with ID starting with fd0bf253af4a07cb1bdfd91a9671f9ed588648726956632a00d0f95ba56884ba not found: ID does not exist"
Oct 30 22:38:37.041955 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:37.041844    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-f56a3efc-58f7fbff8f-s79w2"]
Oct 30 22:38:37.045886 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:37.045848    4473 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-f56a3efc-58f7fbff8f-s79w2"]
Oct 30 22:38:37.156133 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:37.156090    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-f56a3efc-65b4458675-ltsnw"]
Oct 30 22:38:37.160345 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:37.160318    4473 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-f56a3efc-65b4458675-ltsnw"]
Oct 30 22:38:37.270545 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:37.270511    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f56a3efc-794d6757cd-mnjm2"]
Oct 30 22:38:37.277759 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:37.277727    4473 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f56a3efc-794d6757cd-mnjm2"]
Oct 30 22:38:38.356123 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:38.356093    4473 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0baff427-0e69-4480-aaf0-efc881e33fed" path="/var/lib/kubelet/pods/0baff427-0e69-4480-aaf0-efc881e33fed/volumes"
Oct 30 22:38:38.356573 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:38.356553    4473 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="91ff8841-1df2-434d-b422-fbf79c40b844" path="/var/lib/kubelet/pods/91ff8841-1df2-434d-b422-fbf79c40b844/volumes"
Oct 30 22:38:38.357000 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:38.356981    4473 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9e5ce512-6e69-40f2-a47b-b9e559489178" path="/var/lib/kubelet/pods/9e5ce512-6e69-40f2-a47b-b9e559489178/volumes"
Oct 30 22:38:40.282726 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:40.282695    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-f56a3efc-openshift-default-588cd5c7c5-qpzwv"]
Oct 30 22:38:40.283116 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:40.282936    4473 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-f56a3efc-openshift-default-588cd5c7c5-qpzwv" podUID="673eabe0-3a0e-4b7e-82f3-1653e4557d5c" containerName="istio-proxy" containerID="cri-o://50f92f0dade44df03903db95007bed03ef2f9f25401c6588803ae35aedc2b11d" gracePeriod=30
Oct 30 22:38:41.194538 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:41.194491    4473 patch_prober.go:29] interesting pod/e2e-ait-f56a3efc-openshift-default-588cd5c7c5-qpzwv container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 22:38:41.194741 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:41.194566    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-f56a3efc-openshift-default-588cd5c7c5-qpzwv" podUID="673eabe0-3a0e-4b7e-82f3-1653e4557d5c" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 22:38:42.024855 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:42.024799    4473 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-authcfg-http-29f146-openshift-default-67b597f9d5-b5szh"]
Oct 30 22:38:42.025421 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:42.025332    4473 cpu_manager.go:414] "containerMap: removing container" podUID="0baff427-0e69-4480-aaf0-efc881e33fed" containerName="payload-processing"
Oct 30 22:38:42.025421 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:42.025349    4473 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0baff427-0e69-4480-aaf0-efc881e33fed" containerName="payload-processing"
Oct 30 22:38:42.025421 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:42.025371    4473 cpu_manager.go:414] "containerMap: removing container" podUID="9e5ce512-6e69-40f2-a47b-b9e559489178" containerName="payload-pre-processing"
Oct 30 22:38:42.025421 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:42.025377    4473 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9e5ce512-6e69-40f2-a47b-b9e559489178" containerName="payload-pre-processing"
Oct 30 22:38:42.025421 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:42.025396    4473 cpu_manager.go:414] "containerMap: removing container" podUID="91ff8841-1df2-434d-b422-fbf79c40b844" containerName="maas-api"
Oct 30 22:38:42.025421 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:42.025401    4473 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="91ff8841-1df2-434d-b422-fbf79c40b844" containerName="maas-api"
Oct 30 22:38:42.026298 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:42.025474    4473 memory_manager.go:365] "RemoveStaleState removing state" podUID="9e5ce512-6e69-40f2-a47b-b9e559489178" containerName="payload-pre-processing"
Oct 30 22:38:42.026298 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:42.025486    4473 memory_manager.go:365] "RemoveStaleState removing state" podUID="91ff8841-1df2-434d-b422-fbf79c40b844" containerName="maas-api"
Oct 30 22:38:42.026298 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:42.025497    4473 memory_manager.go:365] "RemoveStaleState removing state" podUID="0baff427-0e69-4480-aaf0-efc881e33fed" containerName="payload-processing"
Oct 30 22:38:42.031076 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:42.031037    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-authcfg-http-29f146-openshift-default-67b597f9d5-b5szh"
Oct 30 22:38:42.090589 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:42.090561    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-authcfg-http-29f146-openshift-default-dockercfg-l979d\""
Oct 30 22:38:42.140977 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:42.140947    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-authcfg-http-29f146-openshift-default-67b597f9d5-b5szh"]
Oct 30 22:38:42.151068 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:42.151038    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5fcca3b7-5872-4a79-9189-b96f8bf6a6e6-istio-envoy\") pod \"e2e-authcfg-http-29f146-openshift-default-67b597f9d5-b5szh\" (UID: \"5fcca3b7-5872-4a79-9189-b96f8bf6a6e6\") " pod="openshift-ingress/e2e-authcfg-http-29f146-openshift-default-67b597f9d5-b5szh"
Oct 30 22:38:42.151244 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:42.151088    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5fcca3b7-5872-4a79-9189-b96f8bf6a6e6-istio-podinfo\") pod \"e2e-authcfg-http-29f146-openshift-default-67b597f9d5-b5szh\" (UID: \"5fcca3b7-5872-4a79-9189-b96f8bf6a6e6\") " pod="openshift-ingress/e2e-authcfg-http-29f146-openshift-default-67b597f9d5-b5szh"
Oct 30 22:38:42.151300 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:42.151235    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5fcca3b7-5872-4a79-9189-b96f8bf6a6e6-workload-socket\") pod \"e2e-authcfg-http-29f146-openshift-default-67b597f9d5-b5szh\" (UID: \"5fcca3b7-5872-4a79-9189-b96f8bf6a6e6\") " pod="openshift-ingress/e2e-authcfg-http-29f146-openshift-default-67b597f9d5-b5szh"
Oct 30 22:38:42.151348 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:42.151307    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5fcca3b7-5872-4a79-9189-b96f8bf6a6e6-workload-certs\") pod \"e2e-authcfg-http-29f146-openshift-default-67b597f9d5-b5szh\" (UID: \"5fcca3b7-5872-4a79-9189-b96f8bf6a6e6\") " pod="openshift-ingress/e2e-authcfg-http-29f146-openshift-default-67b597f9d5-b5szh"
Oct 30 22:38:42.151396 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:42.151352    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5fcca3b7-5872-4a79-9189-b96f8bf6a6e6-credential-socket\") pod \"e2e-authcfg-http-29f146-openshift-default-67b597f9d5-b5szh\" (UID: \"5fcca3b7-5872-4a79-9189-b96f8bf6a6e6\") " pod="openshift-ingress/e2e-authcfg-http-29f146-openshift-default-67b597f9d5-b5szh"
Oct 30 22:38:42.151396 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:42.151385    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5fcca3b7-5872-4a79-9189-b96f8bf6a6e6-istio-token\") pod \"e2e-authcfg-http-29f146-openshift-default-67b597f9d5-b5szh\" (UID: \"5fcca3b7-5872-4a79-9189-b96f8bf6a6e6\") " pod="openshift-ingress/e2e-authcfg-http-29f146-openshift-default-67b597f9d5-b5szh"
Oct 30 22:38:42.151501 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:42.151484    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5fcca3b7-5872-4a79-9189-b96f8bf6a6e6-istiod-ca-cert\") pod \"e2e-authcfg-http-29f146-openshift-default-67b597f9d5-b5szh\" (UID: \"5fcca3b7-5872-4a79-9189-b96f8bf6a6e6\") " pod="openshift-ingress/e2e-authcfg-http-29f146-openshift-default-67b597f9d5-b5szh"
Oct 30 22:38:42.151559 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:42.151542    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f67f4\" (UniqueName: \"kubernetes.io/projected/5fcca3b7-5872-4a79-9189-b96f8bf6a6e6-kube-api-access-f67f4\") pod \"e2e-authcfg-http-29f146-openshift-default-67b597f9d5-b5szh\" (UID: \"5fcca3b7-5872-4a79-9189-b96f8bf6a6e6\") " pod="openshift-ingress/e2e-authcfg-http-29f146-openshift-default-67b597f9d5-b5szh"
Oct 30 22:38:42.151603 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:42.151584    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5fcca3b7-5872-4a79-9189-b96f8bf6a6e6-istio-data\") pod \"e2e-authcfg-http-29f146-openshift-default-67b597f9d5-b5szh\" (UID: \"5fcca3b7-5872-4a79-9189-b96f8bf6a6e6\") " pod="openshift-ingress/e2e-authcfg-http-29f146-openshift-default-67b597f9d5-b5szh"
Oct 30 22:38:42.252570 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:42.252532    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5fcca3b7-5872-4a79-9189-b96f8bf6a6e6-istio-envoy\") pod \"e2e-authcfg-http-29f146-openshift-default-67b597f9d5-b5szh\" (UID: \"5fcca3b7-5872-4a79-9189-b96f8bf6a6e6\") " pod="openshift-ingress/e2e-authcfg-http-29f146-openshift-default-67b597f9d5-b5szh"
Oct 30 22:38:42.252570 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:42.252574    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5fcca3b7-5872-4a79-9189-b96f8bf6a6e6-istio-podinfo\") pod \"e2e-authcfg-http-29f146-openshift-default-67b597f9d5-b5szh\" (UID: \"5fcca3b7-5872-4a79-9189-b96f8bf6a6e6\") " pod="openshift-ingress/e2e-authcfg-http-29f146-openshift-default-67b597f9d5-b5szh"
Oct 30 22:38:42.252831 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:42.252614    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5fcca3b7-5872-4a79-9189-b96f8bf6a6e6-workload-socket\") pod \"e2e-authcfg-http-29f146-openshift-default-67b597f9d5-b5szh\" (UID: \"5fcca3b7-5872-4a79-9189-b96f8bf6a6e6\") " pod="openshift-ingress/e2e-authcfg-http-29f146-openshift-default-67b597f9d5-b5szh"
Oct 30 22:38:42.252831 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:42.252645    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5fcca3b7-5872-4a79-9189-b96f8bf6a6e6-workload-certs\") pod \"e2e-authcfg-http-29f146-openshift-default-67b597f9d5-b5szh\" (UID: \"5fcca3b7-5872-4a79-9189-b96f8bf6a6e6\") " pod="openshift-ingress/e2e-authcfg-http-29f146-openshift-default-67b597f9d5-b5szh"
Oct 30 22:38:42.252831 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:42.252674    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5fcca3b7-5872-4a79-9189-b96f8bf6a6e6-credential-socket\") pod \"e2e-authcfg-http-29f146-openshift-default-67b597f9d5-b5szh\" (UID: \"5fcca3b7-5872-4a79-9189-b96f8bf6a6e6\") " pod="openshift-ingress/e2e-authcfg-http-29f146-openshift-default-67b597f9d5-b5szh"
Oct 30 22:38:42.252831 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:42.252705    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5fcca3b7-5872-4a79-9189-b96f8bf6a6e6-istio-token\") pod \"e2e-authcfg-http-29f146-openshift-default-67b597f9d5-b5szh\" (UID: \"5fcca3b7-5872-4a79-9189-b96f8bf6a6e6\") " pod="openshift-ingress/e2e-authcfg-http-29f146-openshift-default-67b597f9d5-b5szh"
Oct 30 22:38:42.252831 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:42.252756    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5fcca3b7-5872-4a79-9189-b96f8bf6a6e6-istiod-ca-cert\") pod \"e2e-authcfg-http-29f146-openshift-default-67b597f9d5-b5szh\" (UID: \"5fcca3b7-5872-4a79-9189-b96f8bf6a6e6\") " pod="openshift-ingress/e2e-authcfg-http-29f146-openshift-default-67b597f9d5-b5szh"
Oct 30 22:38:42.252831 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:42.252802    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f67f4\" (UniqueName: \"kubernetes.io/projected/5fcca3b7-5872-4a79-9189-b96f8bf6a6e6-kube-api-access-f67f4\") pod \"e2e-authcfg-http-29f146-openshift-default-67b597f9d5-b5szh\" (UID: \"5fcca3b7-5872-4a79-9189-b96f8bf6a6e6\") " pod="openshift-ingress/e2e-authcfg-http-29f146-openshift-default-67b597f9d5-b5szh"
Oct 30 22:38:42.253441 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:42.252888    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5fcca3b7-5872-4a79-9189-b96f8bf6a6e6-istio-data\") pod \"e2e-authcfg-http-29f146-openshift-default-67b597f9d5-b5szh\" (UID: \"5fcca3b7-5872-4a79-9189-b96f8bf6a6e6\") " pod="openshift-ingress/e2e-authcfg-http-29f146-openshift-default-67b597f9d5-b5szh"
Oct 30 22:38:42.253441 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:42.253105    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5fcca3b7-5872-4a79-9189-b96f8bf6a6e6-workload-socket\") pod \"e2e-authcfg-http-29f146-openshift-default-67b597f9d5-b5szh\" (UID: \"5fcca3b7-5872-4a79-9189-b96f8bf6a6e6\") " pod="openshift-ingress/e2e-authcfg-http-29f146-openshift-default-67b597f9d5-b5szh"
Oct 30 22:38:42.253554 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:42.253477    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5fcca3b7-5872-4a79-9189-b96f8bf6a6e6-workload-certs\") pod \"e2e-authcfg-http-29f146-openshift-default-67b597f9d5-b5szh\" (UID: \"5fcca3b7-5872-4a79-9189-b96f8bf6a6e6\") " pod="openshift-ingress/e2e-authcfg-http-29f146-openshift-default-67b597f9d5-b5szh"
Oct 30 22:38:42.253554 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:42.253523    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5fcca3b7-5872-4a79-9189-b96f8bf6a6e6-istiod-ca-cert\") pod \"e2e-authcfg-http-29f146-openshift-default-67b597f9d5-b5szh\" (UID: \"5fcca3b7-5872-4a79-9189-b96f8bf6a6e6\") " pod="openshift-ingress/e2e-authcfg-http-29f146-openshift-default-67b597f9d5-b5szh"
Oct 30 22:38:42.253751 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:42.253730    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5fcca3b7-5872-4a79-9189-b96f8bf6a6e6-istio-data\") pod \"e2e-authcfg-http-29f146-openshift-default-67b597f9d5-b5szh\" (UID: \"5fcca3b7-5872-4a79-9189-b96f8bf6a6e6\") " pod="openshift-ingress/e2e-authcfg-http-29f146-openshift-default-67b597f9d5-b5szh"
Oct 30 22:38:42.253970 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:42.253907    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5fcca3b7-5872-4a79-9189-b96f8bf6a6e6-credential-socket\") pod \"e2e-authcfg-http-29f146-openshift-default-67b597f9d5-b5szh\" (UID: \"5fcca3b7-5872-4a79-9189-b96f8bf6a6e6\") " pod="openshift-ingress/e2e-authcfg-http-29f146-openshift-default-67b597f9d5-b5szh"
Oct 30 22:38:42.255302 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:42.255281    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5fcca3b7-5872-4a79-9189-b96f8bf6a6e6-istio-envoy\") pod \"e2e-authcfg-http-29f146-openshift-default-67b597f9d5-b5szh\" (UID: \"5fcca3b7-5872-4a79-9189-b96f8bf6a6e6\") " pod="openshift-ingress/e2e-authcfg-http-29f146-openshift-default-67b597f9d5-b5szh"
Oct 30 22:38:42.255547 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:42.255527    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5fcca3b7-5872-4a79-9189-b96f8bf6a6e6-istio-podinfo\") pod \"e2e-authcfg-http-29f146-openshift-default-67b597f9d5-b5szh\" (UID: \"5fcca3b7-5872-4a79-9189-b96f8bf6a6e6\") " pod="openshift-ingress/e2e-authcfg-http-29f146-openshift-default-67b597f9d5-b5szh"
Oct 30 22:38:42.311075 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:42.310999    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5fcca3b7-5872-4a79-9189-b96f8bf6a6e6-istio-token\") pod \"e2e-authcfg-http-29f146-openshift-default-67b597f9d5-b5szh\" (UID: \"5fcca3b7-5872-4a79-9189-b96f8bf6a6e6\") " pod="openshift-ingress/e2e-authcfg-http-29f146-openshift-default-67b597f9d5-b5szh"
Oct 30 22:38:42.311238 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:42.311190    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f67f4\" (UniqueName: \"kubernetes.io/projected/5fcca3b7-5872-4a79-9189-b96f8bf6a6e6-kube-api-access-f67f4\") pod \"e2e-authcfg-http-29f146-openshift-default-67b597f9d5-b5szh\" (UID: \"5fcca3b7-5872-4a79-9189-b96f8bf6a6e6\") " pod="openshift-ingress/e2e-authcfg-http-29f146-openshift-default-67b597f9d5-b5szh"
Oct 30 22:38:42.345741 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:42.345709    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-authcfg-http-29f146-openshift-default-67b597f9d5-b5szh"
Oct 30 22:38:42.769252 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:42.769224    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-authcfg-http-29f146-openshift-default-67b597f9d5-b5szh"]
Oct 30 22:38:42.771370 ip-10-0-140-18 kubenswrapper[4473]: W1030 22:38:42.771336    4473 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5fcca3b7_5872_4a79_9189_b96f8bf6a6e6.slice/crio-d17642ea1f4a3292d792bd37380a0e27e119aa65d6747c8bf20e82f23f8a8ffa WatchSource:0}: Error finding container d17642ea1f4a3292d792bd37380a0e27e119aa65d6747c8bf20e82f23f8a8ffa: Status 404 returned error can't find the container with id d17642ea1f4a3292d792bd37380a0e27e119aa65d6747c8bf20e82f23f8a8ffa
Oct 30 22:38:42.773492 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:42.773461    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:38:42.773563 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:42.773526    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:38:42.773563 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:42.773557    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:38:42.911619 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:42.911586    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-authcfg-http-29f146-openshift-default-67b597f9d5-b5szh" event={"ID":"5fcca3b7-5872-4a79-9189-b96f8bf6a6e6","Type":"ContainerStarted","Data":"fb6077a9a88a59545c393e6ac816011f9795140311ddf45e529ff38b7009853e"}
Oct 30 22:38:42.911808 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:42.911635    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-authcfg-http-29f146-openshift-default-67b597f9d5-b5szh" event={"ID":"5fcca3b7-5872-4a79-9189-b96f8bf6a6e6","Type":"ContainerStarted","Data":"d17642ea1f4a3292d792bd37380a0e27e119aa65d6747c8bf20e82f23f8a8ffa"}
Oct 30 22:38:43.031378 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:43.031277    4473 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-authcfg-http-29f146-openshift-default-67b597f9d5-b5szh" podStartSLOduration=2.031260748 podStartE2EDuration="2.031260748s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:38:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:38:43.029690269 +0000 UTC m=+3368.803058769" watchObservedRunningTime="2026-10-30 22:38:43.031260748 +0000 UTC m=+3368.804629234"
Oct 30 22:38:43.346963 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:43.346834    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-authcfg-http-29f146-openshift-default-67b597f9d5-b5szh"
Oct 30 22:38:43.352131 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:43.352104    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-authcfg-http-29f146-openshift-default-67b597f9d5-b5szh"
Oct 30 22:38:43.916801 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:43.916772    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-authcfg-http-29f146-openshift-default-67b597f9d5-b5szh"
Oct 30 22:38:43.917814 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:43.917797    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-authcfg-http-29f146-openshift-default-67b597f9d5-b5szh"
Oct 30 22:38:45.537169 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:45.537139    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-f56a3efc-openshift-default-588cd5c7c5-qpzwv"
Oct 30 22:38:45.688606 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:45.688569    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/673eabe0-3a0e-4b7e-82f3-1653e4557d5c-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/673eabe0-3a0e-4b7e-82f3-1653e4557d5c-credential-socket\") pod \"673eabe0-3a0e-4b7e-82f3-1653e4557d5c\" (UID: \"673eabe0-3a0e-4b7e-82f3-1653e4557d5c\") "
Oct 30 22:38:45.688803 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:45.688627    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/673eabe0-3a0e-4b7e-82f3-1653e4557d5c-istio-token\" (UniqueName: \"kubernetes.io/projected/673eabe0-3a0e-4b7e-82f3-1653e4557d5c-istio-token\") pod \"673eabe0-3a0e-4b7e-82f3-1653e4557d5c\" (UID: \"673eabe0-3a0e-4b7e-82f3-1653e4557d5c\") "
Oct 30 22:38:45.688803 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:45.688678    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/673eabe0-3a0e-4b7e-82f3-1653e4557d5c-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/673eabe0-3a0e-4b7e-82f3-1653e4557d5c-istio-data\") pod \"673eabe0-3a0e-4b7e-82f3-1653e4557d5c\" (UID: \"673eabe0-3a0e-4b7e-82f3-1653e4557d5c\") "
Oct 30 22:38:45.688803 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:45.688703    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/673eabe0-3a0e-4b7e-82f3-1653e4557d5c-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/673eabe0-3a0e-4b7e-82f3-1653e4557d5c-istio-podinfo\") pod \"673eabe0-3a0e-4b7e-82f3-1653e4557d5c\" (UID: \"673eabe0-3a0e-4b7e-82f3-1653e4557d5c\") "
Oct 30 22:38:45.688803 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:45.688768    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/673eabe0-3a0e-4b7e-82f3-1653e4557d5c-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/673eabe0-3a0e-4b7e-82f3-1653e4557d5c-istiod-ca-cert\") pod \"673eabe0-3a0e-4b7e-82f3-1653e4557d5c\" (UID: \"673eabe0-3a0e-4b7e-82f3-1653e4557d5c\") "
Oct 30 22:38:45.689060 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:45.688822    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/673eabe0-3a0e-4b7e-82f3-1653e4557d5c-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/673eabe0-3a0e-4b7e-82f3-1653e4557d5c-workload-socket\") pod \"673eabe0-3a0e-4b7e-82f3-1653e4557d5c\" (UID: \"673eabe0-3a0e-4b7e-82f3-1653e4557d5c\") "
Oct 30 22:38:45.689060 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:45.688876    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/673eabe0-3a0e-4b7e-82f3-1653e4557d5c-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/673eabe0-3a0e-4b7e-82f3-1653e4557d5c-workload-certs\") pod \"673eabe0-3a0e-4b7e-82f3-1653e4557d5c\" (UID: \"673eabe0-3a0e-4b7e-82f3-1653e4557d5c\") "
Oct 30 22:38:45.689060 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:45.688887    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/673eabe0-3a0e-4b7e-82f3-1653e4557d5c-credential-socket" pod "673eabe0-3a0e-4b7e-82f3-1653e4557d5c" (UID: "673eabe0-3a0e-4b7e-82f3-1653e4557d5c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:38:45.689060 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:45.689031    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/673eabe0-3a0e-4b7e-82f3-1653e4557d5c-istio-data" pod "673eabe0-3a0e-4b7e-82f3-1653e4557d5c" (UID: "673eabe0-3a0e-4b7e-82f3-1653e4557d5c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:38:45.689296 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:45.689117    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/673eabe0-3a0e-4b7e-82f3-1653e4557d5c-workload-socket" pod "673eabe0-3a0e-4b7e-82f3-1653e4557d5c" (UID: "673eabe0-3a0e-4b7e-82f3-1653e4557d5c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:38:45.689296 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:45.689133    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/673eabe0-3a0e-4b7e-82f3-1653e4557d5c-kube-api-access-pntsd\" (UniqueName: \"kubernetes.io/projected/673eabe0-3a0e-4b7e-82f3-1653e4557d5c-kube-api-access-pntsd\") pod \"673eabe0-3a0e-4b7e-82f3-1653e4557d5c\" (UID: \"673eabe0-3a0e-4b7e-82f3-1653e4557d5c\") "
Oct 30 22:38:45.689296 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:45.689146    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/673eabe0-3a0e-4b7e-82f3-1653e4557d5c-workload-certs" pod "673eabe0-3a0e-4b7e-82f3-1653e4557d5c" (UID: "673eabe0-3a0e-4b7e-82f3-1653e4557d5c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:38:45.689296 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:45.689168    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/673eabe0-3a0e-4b7e-82f3-1653e4557d5c-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/673eabe0-3a0e-4b7e-82f3-1653e4557d5c-istio-envoy\") pod \"673eabe0-3a0e-4b7e-82f3-1653e4557d5c\" (UID: \"673eabe0-3a0e-4b7e-82f3-1653e4557d5c\") "
Oct 30 22:38:45.689296 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:45.689234    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/673eabe0-3a0e-4b7e-82f3-1653e4557d5c-istiod-ca-cert" pod "673eabe0-3a0e-4b7e-82f3-1653e4557d5c" (UID: "673eabe0-3a0e-4b7e-82f3-1653e4557d5c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:38:45.689714 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:45.689576    4473 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/673eabe0-3a0e-4b7e-82f3-1653e4557d5c-istiod-ca-cert\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:38:45.689714 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:45.689600    4473 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/673eabe0-3a0e-4b7e-82f3-1653e4557d5c-workload-socket\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:38:45.689714 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:45.689613    4473 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/673eabe0-3a0e-4b7e-82f3-1653e4557d5c-workload-certs\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:38:45.689714 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:45.689623    4473 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/673eabe0-3a0e-4b7e-82f3-1653e4557d5c-credential-socket\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:38:45.689714 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:45.689632    4473 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/673eabe0-3a0e-4b7e-82f3-1653e4557d5c-istio-data\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:38:45.691173 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:45.691140    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/673eabe0-3a0e-4b7e-82f3-1653e4557d5c-istio-token" pod "673eabe0-3a0e-4b7e-82f3-1653e4557d5c" (UID: "673eabe0-3a0e-4b7e-82f3-1653e4557d5c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:38:45.691173 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:45.691168    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/673eabe0-3a0e-4b7e-82f3-1653e4557d5c-istio-podinfo" pod "673eabe0-3a0e-4b7e-82f3-1653e4557d5c" (UID: "673eabe0-3a0e-4b7e-82f3-1653e4557d5c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:38:45.691800 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:45.691783    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/673eabe0-3a0e-4b7e-82f3-1653e4557d5c-istio-envoy" pod "673eabe0-3a0e-4b7e-82f3-1653e4557d5c" (UID: "673eabe0-3a0e-4b7e-82f3-1653e4557d5c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:38:45.692018 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:45.691814    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/673eabe0-3a0e-4b7e-82f3-1653e4557d5c-kube-api-access-pntsd" pod "673eabe0-3a0e-4b7e-82f3-1653e4557d5c" (UID: "673eabe0-3a0e-4b7e-82f3-1653e4557d5c"). InnerVolumeSpecName "kube-api-access-pntsd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:38:45.791019 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:45.790982    4473 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pntsd\" (UniqueName: \"kubernetes.io/projected/673eabe0-3a0e-4b7e-82f3-1653e4557d5c-kube-api-access-pntsd\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:38:45.791019 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:45.791016    4473 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/673eabe0-3a0e-4b7e-82f3-1653e4557d5c-istio-envoy\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:38:45.791019 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:45.791026    4473 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/673eabe0-3a0e-4b7e-82f3-1653e4557d5c-istio-token\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:38:45.791273 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:45.791036    4473 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/673eabe0-3a0e-4b7e-82f3-1653e4557d5c-istio-podinfo\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:38:45.928347 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:45.928308    4473 generic.go:356] "Generic (PLEG): container finished" podID="673eabe0-3a0e-4b7e-82f3-1653e4557d5c" containerID="50f92f0dade44df03903db95007bed03ef2f9f25401c6588803ae35aedc2b11d" exitCode=0
Oct 30 22:38:45.928499 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:45.928360    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f56a3efc-openshift-default-588cd5c7c5-qpzwv" event={"ID":"673eabe0-3a0e-4b7e-82f3-1653e4557d5c","Type":"ContainerDied","Data":"50f92f0dade44df03903db95007bed03ef2f9f25401c6588803ae35aedc2b11d"}
Oct 30 22:38:45.928499 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:45.928383    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-f56a3efc-openshift-default-588cd5c7c5-qpzwv"
Oct 30 22:38:45.928499 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:45.928440    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f56a3efc-openshift-default-588cd5c7c5-qpzwv" event={"ID":"673eabe0-3a0e-4b7e-82f3-1653e4557d5c","Type":"ContainerDied","Data":"a177c187f53a4a4020554ddbf9bc8becf30859c8cc9cbdb66791302ded5d32d7"}
Oct 30 22:38:45.928499 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:45.928462    4473 scope.go:122] "RemoveContainer" containerID="50f92f0dade44df03903db95007bed03ef2f9f25401c6588803ae35aedc2b11d"
Oct 30 22:38:45.938695 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:45.938636    4473 scope.go:122] "RemoveContainer" containerID="e2ae4fabb1e5817b1f14f6f8761f587fdf5e876344e6c48d6d6ca1eecd944071"
Oct 30 22:38:45.949568 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:45.949542    4473 scope.go:122] "RemoveContainer" containerID="50f92f0dade44df03903db95007bed03ef2f9f25401c6588803ae35aedc2b11d"
Oct 30 22:38:45.950114 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:38:45.950089    4473 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"50f92f0dade44df03903db95007bed03ef2f9f25401c6588803ae35aedc2b11d\": container with ID starting with 50f92f0dade44df03903db95007bed03ef2f9f25401c6588803ae35aedc2b11d not found: ID does not exist" containerID="50f92f0dade44df03903db95007bed03ef2f9f25401c6588803ae35aedc2b11d"
Oct 30 22:38:45.950235 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:45.950128    4473 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"50f92f0dade44df03903db95007bed03ef2f9f25401c6588803ae35aedc2b11d"} err="failed to get container status \"50f92f0dade44df03903db95007bed03ef2f9f25401c6588803ae35aedc2b11d\": rpc error: code = NotFound desc = could not find container \"50f92f0dade44df03903db95007bed03ef2f9f25401c6588803ae35aedc2b11d\": container with ID starting with 50f92f0dade44df03903db95007bed03ef2f9f25401c6588803ae35aedc2b11d not found: ID does not exist"
Oct 30 22:38:45.950235 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:45.950153    4473 scope.go:122] "RemoveContainer" containerID="e2ae4fabb1e5817b1f14f6f8761f587fdf5e876344e6c48d6d6ca1eecd944071"
Oct 30 22:38:45.950480 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:38:45.950458    4473 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e2ae4fabb1e5817b1f14f6f8761f587fdf5e876344e6c48d6d6ca1eecd944071\": container with ID starting with e2ae4fabb1e5817b1f14f6f8761f587fdf5e876344e6c48d6d6ca1eecd944071 not found: ID does not exist" containerID="e2ae4fabb1e5817b1f14f6f8761f587fdf5e876344e6c48d6d6ca1eecd944071"
Oct 30 22:38:45.950534 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:45.950489    4473 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e2ae4fabb1e5817b1f14f6f8761f587fdf5e876344e6c48d6d6ca1eecd944071"} err="failed to get container status \"e2ae4fabb1e5817b1f14f6f8761f587fdf5e876344e6c48d6d6ca1eecd944071\": rpc error: code = NotFound desc = could not find container \"e2ae4fabb1e5817b1f14f6f8761f587fdf5e876344e6c48d6d6ca1eecd944071\": container with ID starting with e2ae4fabb1e5817b1f14f6f8761f587fdf5e876344e6c48d6d6ca1eecd944071 not found: ID does not exist"
Oct 30 22:38:46.116712 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:46.116678    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-f56a3efc-openshift-default-588cd5c7c5-qpzwv"]
Oct 30 22:38:46.166804 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:46.166771    4473 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-f56a3efc-openshift-default-588cd5c7c5-qpzwv"]
Oct 30 22:38:46.356060 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:46.355987    4473 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="673eabe0-3a0e-4b7e-82f3-1653e4557d5c" path="/var/lib/kubelet/pods/673eabe0-3a0e-4b7e-82f3-1653e4557d5c/volumes"
Oct 30 22:38:49.438143 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:49.438108    4473 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-authcfg-http-29f146-75787ff4dc-wvbh7"]
Oct 30 22:38:49.438548 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:49.438537    4473 cpu_manager.go:414] "containerMap: removing container" podUID="673eabe0-3a0e-4b7e-82f3-1653e4557d5c" containerName="istio-proxy"
Oct 30 22:38:49.438589 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:49.438550    4473 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="673eabe0-3a0e-4b7e-82f3-1653e4557d5c" containerName="istio-proxy"
Oct 30 22:38:49.438589 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:49.438563    4473 cpu_manager.go:414] "containerMap: removing container" podUID="673eabe0-3a0e-4b7e-82f3-1653e4557d5c" containerName="istio-proxy"
Oct 30 22:38:49.438589 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:49.438568    4473 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="673eabe0-3a0e-4b7e-82f3-1653e4557d5c" containerName="istio-proxy"
Oct 30 22:38:49.438688 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:49.438640    4473 memory_manager.go:365] "RemoveStaleState removing state" podUID="673eabe0-3a0e-4b7e-82f3-1653e4557d5c" containerName="istio-proxy"
Oct 30 22:38:49.443104 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:49.443083    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-29f146-75787ff4dc-wvbh7"
Oct 30 22:38:49.530975 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:49.530895    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/42a9e6b6-88e5-477a-8202-512329baa28e-plugins-config-volume\") pod \"payload-pre-processing-e2e-authcfg-http-29f146-75787ff4dc-wvbh7\" (UID: \"42a9e6b6-88e5-477a-8202-512329baa28e\") " pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-29f146-75787ff4dc-wvbh7"
Oct 30 22:38:49.531602 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:49.531569    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j4tj6\" (UniqueName: \"kubernetes.io/projected/42a9e6b6-88e5-477a-8202-512329baa28e-kube-api-access-j4tj6\") pod \"payload-pre-processing-e2e-authcfg-http-29f146-75787ff4dc-wvbh7\" (UID: \"42a9e6b6-88e5-477a-8202-512329baa28e\") " pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-29f146-75787ff4dc-wvbh7"
Oct 30 22:38:49.547421 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:49.547387    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-authcfg-http-29f146-dockercfg-47nf8\""
Oct 30 22:38:49.547421 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:49.547389    4473 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-authcfg-http-29f146\""
Oct 30 22:38:49.549469 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:49.549444    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-authcfg-http-29f146-75787ff4dc-wvbh7"]
Oct 30 22:38:49.588541 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:49.588510    4473 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-authcfg-http-29f146-6fc88476bc-bnvws"]
Oct 30 22:38:49.589821 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:49.589792    4473 memory_manager.go:365] "RemoveStaleState removing state" podUID="673eabe0-3a0e-4b7e-82f3-1653e4557d5c" containerName="istio-proxy"
Oct 30 22:38:49.593006 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:49.592985    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-authcfg-http-29f146-6fc88476bc-bnvws"
Oct 30 22:38:49.632671 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:49.632636    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/42a9e6b6-88e5-477a-8202-512329baa28e-plugins-config-volume\") pod \"payload-pre-processing-e2e-authcfg-http-29f146-75787ff4dc-wvbh7\" (UID: \"42a9e6b6-88e5-477a-8202-512329baa28e\") " pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-29f146-75787ff4dc-wvbh7"
Oct 30 22:38:49.632894 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:49.632730    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j4tj6\" (UniqueName: \"kubernetes.io/projected/42a9e6b6-88e5-477a-8202-512329baa28e-kube-api-access-j4tj6\") pod \"payload-pre-processing-e2e-authcfg-http-29f146-75787ff4dc-wvbh7\" (UID: \"42a9e6b6-88e5-477a-8202-512329baa28e\") " pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-29f146-75787ff4dc-wvbh7"
Oct 30 22:38:49.633436 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:49.633412    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/42a9e6b6-88e5-477a-8202-512329baa28e-plugins-config-volume\") pod \"payload-pre-processing-e2e-authcfg-http-29f146-75787ff4dc-wvbh7\" (UID: \"42a9e6b6-88e5-477a-8202-512329baa28e\") " pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-29f146-75787ff4dc-wvbh7"
Oct 30 22:38:49.649267 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:49.649228    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-authcfg-http-29f146-6fc88476bc-bnvws"]
Oct 30 22:38:49.691350 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:49.691270    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j4tj6\" (UniqueName: \"kubernetes.io/projected/42a9e6b6-88e5-477a-8202-512329baa28e-kube-api-access-j4tj6\") pod \"payload-pre-processing-e2e-authcfg-http-29f146-75787ff4dc-wvbh7\" (UID: \"42a9e6b6-88e5-477a-8202-512329baa28e\") " pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-29f146-75787ff4dc-wvbh7"
Oct 30 22:38:49.733771 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:49.733728    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8e61f8d9-d9f6-4b68-9e6b-434cc58f4f78-plugins-config-volume\") pod \"payload-processing-e2e-authcfg-http-29f146-6fc88476bc-bnvws\" (UID: \"8e61f8d9-d9f6-4b68-9e6b-434cc58f4f78\") " pod="openshift-ingress/payload-processing-e2e-authcfg-http-29f146-6fc88476bc-bnvws"
Oct 30 22:38:49.733976 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:49.733887    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fbq69\" (UniqueName: \"kubernetes.io/projected/8e61f8d9-d9f6-4b68-9e6b-434cc58f4f78-kube-api-access-fbq69\") pod \"payload-processing-e2e-authcfg-http-29f146-6fc88476bc-bnvws\" (UID: \"8e61f8d9-d9f6-4b68-9e6b-434cc58f4f78\") " pod="openshift-ingress/payload-processing-e2e-authcfg-http-29f146-6fc88476bc-bnvws"
Oct 30 22:38:49.756410 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:49.756371    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-29f146-75787ff4dc-wvbh7"
Oct 30 22:38:49.838958 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:49.835246    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8e61f8d9-d9f6-4b68-9e6b-434cc58f4f78-plugins-config-volume\") pod \"payload-processing-e2e-authcfg-http-29f146-6fc88476bc-bnvws\" (UID: \"8e61f8d9-d9f6-4b68-9e6b-434cc58f4f78\") " pod="openshift-ingress/payload-processing-e2e-authcfg-http-29f146-6fc88476bc-bnvws"
Oct 30 22:38:49.838958 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:49.835346    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fbq69\" (UniqueName: \"kubernetes.io/projected/8e61f8d9-d9f6-4b68-9e6b-434cc58f4f78-kube-api-access-fbq69\") pod \"payload-processing-e2e-authcfg-http-29f146-6fc88476bc-bnvws\" (UID: \"8e61f8d9-d9f6-4b68-9e6b-434cc58f4f78\") " pod="openshift-ingress/payload-processing-e2e-authcfg-http-29f146-6fc88476bc-bnvws"
Oct 30 22:38:49.838958 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:49.836278    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8e61f8d9-d9f6-4b68-9e6b-434cc58f4f78-plugins-config-volume\") pod \"payload-processing-e2e-authcfg-http-29f146-6fc88476bc-bnvws\" (UID: \"8e61f8d9-d9f6-4b68-9e6b-434cc58f4f78\") " pod="openshift-ingress/payload-processing-e2e-authcfg-http-29f146-6fc88476bc-bnvws"
Oct 30 22:38:49.893895 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:49.893866    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fbq69\" (UniqueName: \"kubernetes.io/projected/8e61f8d9-d9f6-4b68-9e6b-434cc58f4f78-kube-api-access-fbq69\") pod \"payload-processing-e2e-authcfg-http-29f146-6fc88476bc-bnvws\" (UID: \"8e61f8d9-d9f6-4b68-9e6b-434cc58f4f78\") " pod="openshift-ingress/payload-processing-e2e-authcfg-http-29f146-6fc88476bc-bnvws"
Oct 30 22:38:49.906751 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:49.906722    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-authcfg-http-29f146-6fc88476bc-bnvws"
Oct 30 22:38:50.130245 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:50.130215    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-authcfg-http-29f146-75787ff4dc-wvbh7"]
Oct 30 22:38:50.131953 ip-10-0-140-18 kubenswrapper[4473]: W1030 22:38:50.131905    4473 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42a9e6b6_88e5_477a_8202_512329baa28e.slice/crio-b80f0a71001e195150019725715cca5892e200bff9a1c10eb93082177b2b5442 WatchSource:0}: Error finding container b80f0a71001e195150019725715cca5892e200bff9a1c10eb93082177b2b5442: Status 404 returned error can't find the container with id b80f0a71001e195150019725715cca5892e200bff9a1c10eb93082177b2b5442
Oct 30 22:38:50.287411 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:50.287380    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-authcfg-http-29f146-6fc88476bc-bnvws"]
Oct 30 22:38:50.288906 ip-10-0-140-18 kubenswrapper[4473]: W1030 22:38:50.288879    4473 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e61f8d9_d9f6_4b68_9e6b_434cc58f4f78.slice/crio-589c94acdd6706336f0d4ebd5d8d897e8226255a7b884a1949af178643f89a97 WatchSource:0}: Error finding container 589c94acdd6706336f0d4ebd5d8d897e8226255a7b884a1949af178643f89a97: Status 404 returned error can't find the container with id 589c94acdd6706336f0d4ebd5d8d897e8226255a7b884a1949af178643f89a97
Oct 30 22:38:50.323306 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:38:50.323271    4473 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fncvg" containerName="extractor"
Oct 30 22:38:50.958755 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:50.958716    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-29f146-75787ff4dc-wvbh7" event={"ID":"42a9e6b6-88e5-477a-8202-512329baa28e","Type":"ContainerStarted","Data":"2c44fb5d5e52ae57ebd405f1ed244ab1b113fe8702e34e29897dfa7ea71cdfab"}
Oct 30 22:38:50.959220 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:50.958781    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-29f146-75787ff4dc-wvbh7" event={"ID":"42a9e6b6-88e5-477a-8202-512329baa28e","Type":"ContainerStarted","Data":"b80f0a71001e195150019725715cca5892e200bff9a1c10eb93082177b2b5442"}
Oct 30 22:38:50.959220 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:50.958900    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-29f146-75787ff4dc-wvbh7"
Oct 30 22:38:50.960304 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:50.960279    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-authcfg-http-29f146-6fc88476bc-bnvws" event={"ID":"8e61f8d9-d9f6-4b68-9e6b-434cc58f4f78","Type":"ContainerStarted","Data":"61db328e2d92e41ba8259a31644204807b8322269c40b885caa2ba0f1b0ded6f"}
Oct 30 22:38:50.960443 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:50.960314    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-authcfg-http-29f146-6fc88476bc-bnvws" event={"ID":"8e61f8d9-d9f6-4b68-9e6b-434cc58f4f78","Type":"ContainerStarted","Data":"589c94acdd6706336f0d4ebd5d8d897e8226255a7b884a1949af178643f89a97"}
Oct 30 22:38:50.960443 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:50.960350    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-authcfg-http-29f146-6fc88476bc-bnvws"
Oct 30 22:38:51.076350 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:51.076294    4473 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-29f146-75787ff4dc-wvbh7" podStartSLOduration=2.07627945 podStartE2EDuration="2.07627945s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:38:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:38:51.075733145 +0000 UTC m=+3376.849101631" watchObservedRunningTime="2026-10-30 22:38:51.07627945 +0000 UTC m=+3376.849647937"
Oct 30 22:38:51.198164 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:51.198098    4473 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-authcfg-http-29f146-6fc88476bc-bnvws" podStartSLOduration=2.1980743719999998 podStartE2EDuration="2.198074372s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:38:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:38:51.191176489 +0000 UTC m=+3376.964544977" watchObservedRunningTime="2026-10-30 22:38:51.198074372 +0000 UTC m=+3376.971442859"
Oct 30 22:38:52.889518 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:52.889474    4473 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-authcfg-http-29f146-759775bdb6-gmjg9"]
Oct 30 22:38:52.893373 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:52.893349    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-29f146-759775bdb6-gmjg9"
Oct 30 22:38:52.946300 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:52.946264    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-authcfg-http-29f146-serving-cert\""
Oct 30 22:38:52.998950 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:52.998898    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-authcfg-http-29f146-759775bdb6-gmjg9"]
Oct 30 22:38:53.069842 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:53.069805    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dccnd\" (UniqueName: \"kubernetes.io/projected/0cde6324-cceb-491c-b40a-50cf2109a542-kube-api-access-dccnd\") pod \"maas-api-e2e-authcfg-http-29f146-759775bdb6-gmjg9\" (UID: \"0cde6324-cceb-491c-b40a-50cf2109a542\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-29f146-759775bdb6-gmjg9"
Oct 30 22:38:53.069842 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:53.069843    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0cde6324-cceb-491c-b40a-50cf2109a542-openshift-service-ca\") pod \"maas-api-e2e-authcfg-http-29f146-759775bdb6-gmjg9\" (UID: \"0cde6324-cceb-491c-b40a-50cf2109a542\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-29f146-759775bdb6-gmjg9"
Oct 30 22:38:53.070131 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:53.070071    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0cde6324-cceb-491c-b40a-50cf2109a542-maas-api-tls\") pod \"maas-api-e2e-authcfg-http-29f146-759775bdb6-gmjg9\" (UID: \"0cde6324-cceb-491c-b40a-50cf2109a542\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-29f146-759775bdb6-gmjg9"
Oct 30 22:38:53.070131 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:53.070121    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0cde6324-cceb-491c-b40a-50cf2109a542-metrics-certs\") pod \"maas-api-e2e-authcfg-http-29f146-759775bdb6-gmjg9\" (UID: \"0cde6324-cceb-491c-b40a-50cf2109a542\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-29f146-759775bdb6-gmjg9"
Oct 30 22:38:53.171607 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:53.171523    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0cde6324-cceb-491c-b40a-50cf2109a542-maas-api-tls\") pod \"maas-api-e2e-authcfg-http-29f146-759775bdb6-gmjg9\" (UID: \"0cde6324-cceb-491c-b40a-50cf2109a542\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-29f146-759775bdb6-gmjg9"
Oct 30 22:38:53.171607 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:53.171563    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0cde6324-cceb-491c-b40a-50cf2109a542-metrics-certs\") pod \"maas-api-e2e-authcfg-http-29f146-759775bdb6-gmjg9\" (UID: \"0cde6324-cceb-491c-b40a-50cf2109a542\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-29f146-759775bdb6-gmjg9"
Oct 30 22:38:53.171854 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:53.171627    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dccnd\" (UniqueName: \"kubernetes.io/projected/0cde6324-cceb-491c-b40a-50cf2109a542-kube-api-access-dccnd\") pod \"maas-api-e2e-authcfg-http-29f146-759775bdb6-gmjg9\" (UID: \"0cde6324-cceb-491c-b40a-50cf2109a542\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-29f146-759775bdb6-gmjg9"
Oct 30 22:38:53.171854 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:53.171653    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0cde6324-cceb-491c-b40a-50cf2109a542-openshift-service-ca\") pod \"maas-api-e2e-authcfg-http-29f146-759775bdb6-gmjg9\" (UID: \"0cde6324-cceb-491c-b40a-50cf2109a542\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-29f146-759775bdb6-gmjg9"
Oct 30 22:38:53.171854 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:38:53.171701    4473 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-authcfg-http-29f146-serving-cert: secret "maas-api-e2e-authcfg-http-29f146-serving-cert" not found
Oct 30 22:38:53.171854 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:38:53.171770    4473 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0cde6324-cceb-491c-b40a-50cf2109a542-maas-api-tls podName:0cde6324-cceb-491c-b40a-50cf2109a542 nodeName:}" failed. No retries permitted until 2026-10-30 22:38:53.671749509 +0000 UTC m=+3379.445117994 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/0cde6324-cceb-491c-b40a-50cf2109a542-maas-api-tls") pod "maas-api-e2e-authcfg-http-29f146-759775bdb6-gmjg9" (UID: "0cde6324-cceb-491c-b40a-50cf2109a542") : secret "maas-api-e2e-authcfg-http-29f146-serving-cert" not found
Oct 30 22:38:53.172389 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:53.172368    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0cde6324-cceb-491c-b40a-50cf2109a542-openshift-service-ca\") pod \"maas-api-e2e-authcfg-http-29f146-759775bdb6-gmjg9\" (UID: \"0cde6324-cceb-491c-b40a-50cf2109a542\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-29f146-759775bdb6-gmjg9"
Oct 30 22:38:53.174395 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:53.174374    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0cde6324-cceb-491c-b40a-50cf2109a542-metrics-certs\") pod \"maas-api-e2e-authcfg-http-29f146-759775bdb6-gmjg9\" (UID: \"0cde6324-cceb-491c-b40a-50cf2109a542\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-29f146-759775bdb6-gmjg9"
Oct 30 22:38:53.229979 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:53.229903    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dccnd\" (UniqueName: \"kubernetes.io/projected/0cde6324-cceb-491c-b40a-50cf2109a542-kube-api-access-dccnd\") pod \"maas-api-e2e-authcfg-http-29f146-759775bdb6-gmjg9\" (UID: \"0cde6324-cceb-491c-b40a-50cf2109a542\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-29f146-759775bdb6-gmjg9"
Oct 30 22:38:53.676395 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:53.676360    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0cde6324-cceb-491c-b40a-50cf2109a542-maas-api-tls\") pod \"maas-api-e2e-authcfg-http-29f146-759775bdb6-gmjg9\" (UID: \"0cde6324-cceb-491c-b40a-50cf2109a542\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-29f146-759775bdb6-gmjg9"
Oct 30 22:38:53.678984 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:53.678957    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0cde6324-cceb-491c-b40a-50cf2109a542-maas-api-tls\") pod \"maas-api-e2e-authcfg-http-29f146-759775bdb6-gmjg9\" (UID: \"0cde6324-cceb-491c-b40a-50cf2109a542\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-29f146-759775bdb6-gmjg9"
Oct 30 22:38:53.806524 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:53.806491    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-29f146-759775bdb6-gmjg9"
Oct 30 22:38:54.186431 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:54.186398    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-authcfg-http-29f146-759775bdb6-gmjg9"]
Oct 30 22:38:54.187282 ip-10-0-140-18 kubenswrapper[4473]: W1030 22:38:54.187256    4473 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0cde6324_cceb_491c_b40a_50cf2109a542.slice/crio-df8c6dcce78898d254eda22ca932cbf51aecb6e6101f438aaa128e7da8ca0c70 WatchSource:0}: Error finding container df8c6dcce78898d254eda22ca932cbf51aecb6e6101f438aaa128e7da8ca0c70: Status 404 returned error can't find the container with id df8c6dcce78898d254eda22ca932cbf51aecb6e6101f438aaa128e7da8ca0c70
Oct 30 22:38:54.986569 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:54.986533    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-29f146-759775bdb6-gmjg9" event={"ID":"0cde6324-cceb-491c-b40a-50cf2109a542","Type":"ContainerStarted","Data":"df8c6dcce78898d254eda22ca932cbf51aecb6e6101f438aaa128e7da8ca0c70"}
Oct 30 22:38:55.993108 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:55.993072    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-29f146-759775bdb6-gmjg9" event={"ID":"0cde6324-cceb-491c-b40a-50cf2109a542","Type":"ContainerStarted","Data":"e35356b3ecfaa58440dd54165a504fa8a5db5a82839a8f47458295fd24504d58"}
Oct 30 22:38:55.993553 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:38:55.993228    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-29f146-759775bdb6-gmjg9"
Oct 30 22:39:01.968797 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:39:01.968766    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-authcfg-http-29f146-6fc88476bc-bnvws"
Oct 30 22:39:01.969283 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:39:01.969176    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-29f146-75787ff4dc-wvbh7"
Oct 30 22:39:02.002993 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:39:02.002966    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-29f146-759775bdb6-gmjg9"
Oct 30 22:39:02.093322 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:39:02.093262    4473 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-29f146-759775bdb6-gmjg9" podStartSLOduration=9.30364303 podStartE2EDuration="10.093247096s" totalImagesPullingTime="789.604066ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:38:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:38:56.110158094 +0000 UTC m=+3381.883526584" watchObservedRunningTime="2026-10-30 22:39:02.093247096 +0000 UTC m=+3387.866615632"
Oct 30 22:39:11.099518 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:39:11.099484    4473 generic.go:356] "Generic (PLEG): container finished" podID="5fcca3b7-5872-4a79-9189-b96f8bf6a6e6" containerID="fb6077a9a88a59545c393e6ac816011f9795140311ddf45e529ff38b7009853e" exitCode=0
Oct 30 22:39:11.100004 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:39:11.099544    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-authcfg-http-29f146-openshift-default-67b597f9d5-b5szh" event={"ID":"5fcca3b7-5872-4a79-9189-b96f8bf6a6e6","Type":"ContainerDied","Data":"fb6077a9a88a59545c393e6ac816011f9795140311ddf45e529ff38b7009853e"}
Oct 30 22:39:11.100004 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:39:11.099991    4473 scope.go:122] "RemoveContainer" containerID="fb6077a9a88a59545c393e6ac816011f9795140311ddf45e529ff38b7009853e"
Oct 30 22:39:11.101368 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:39:11.101337    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:39:11.101439 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:39:11.101406    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:39:11.101477 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:39:11.101437    4473 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 22:39:12.107507 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:39:12.107469    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-authcfg-http-29f146-openshift-default-67b597f9d5-b5szh" event={"ID":"5fcca3b7-5872-4a79-9189-b96f8bf6a6e6","Type":"ContainerStarted","Data":"7c79f24eebd86c6a4abd049f1a6d8a74430ae8af4532cf7bbb04d736c9dd2adc"}
Oct 30 22:39:12.348219 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:39:12.348115    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-authcfg-http-29f146-openshift-default-67b597f9d5-b5szh"
Oct 30 22:39:12.350128 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:39:12.350064    4473 patch_prober.go:29] interesting pod/e2e-authcfg-http-29f146-openshift-default-67b597f9d5-b5szh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.167:15021/healthz/ready\": dial tcp 10.133.0.167:15021: connect: connection refused" start-of-body=
Oct 30 22:39:12.350308 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:39:12.350160    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-authcfg-http-29f146-openshift-default-67b597f9d5-b5szh" podUID="5fcca3b7-5872-4a79-9189-b96f8bf6a6e6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.167:15021/healthz/ready\": dial tcp 10.133.0.167:15021: connect: connection refused"
Oct 30 22:39:13.350399 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:39:13.350348    4473 patch_prober.go:29] interesting pod/e2e-authcfg-http-29f146-openshift-default-67b597f9d5-b5szh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.167:15021/healthz/ready\": dial tcp 10.133.0.167:15021: connect: connection refused" start-of-body=
Oct 30 22:39:13.350822 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:39:13.350423    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-authcfg-http-29f146-openshift-default-67b597f9d5-b5szh" podUID="5fcca3b7-5872-4a79-9189-b96f8bf6a6e6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.167:15021/healthz/ready\": dial tcp 10.133.0.167:15021: connect: connection refused"
Oct 30 22:39:13.917080 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:39:13.917035    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-authcfg-http-29f146-openshift-default-67b597f9d5-b5szh"
Oct 30 22:39:14.346531 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:39:14.346433    4473 patch_prober.go:29] interesting pod/e2e-authcfg-http-29f146-openshift-default-67b597f9d5-b5szh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.167:15021/healthz/ready\": dial tcp 10.133.0.167:15021: connect: connection refused" start-of-body=
Oct 30 22:39:14.346821 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:39:14.346787    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-authcfg-http-29f146-openshift-default-67b597f9d5-b5szh" podUID="5fcca3b7-5872-4a79-9189-b96f8bf6a6e6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.167:15021/healthz/ready\": dial tcp 10.133.0.167:15021: connect: connection refused"
Oct 30 22:39:15.346395 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:39:15.346350    4473 patch_prober.go:29] interesting pod/e2e-authcfg-http-29f146-openshift-default-67b597f9d5-b5szh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.167:15021/healthz/ready\": dial tcp 10.133.0.167:15021: connect: connection refused" start-of-body=
Oct 30 22:39:15.347083 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:39:15.347041    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-authcfg-http-29f146-openshift-default-67b597f9d5-b5szh" podUID="5fcca3b7-5872-4a79-9189-b96f8bf6a6e6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.167:15021/healthz/ready\": dial tcp 10.133.0.167:15021: connect: connection refused"
Oct 30 22:39:16.351685 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:39:16.351654    4473 patch_prober.go:29] interesting pod/e2e-authcfg-http-29f146-openshift-default-67b597f9d5-b5szh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.167:15021/healthz/ready\": dial tcp 10.133.0.167:15021: connect: connection refused" start-of-body=
Oct 30 22:39:16.352195 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:39:16.352170    4473 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-authcfg-http-29f146-openshift-default-67b597f9d5-b5szh" podUID="5fcca3b7-5872-4a79-9189-b96f8bf6a6e6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.167:15021/healthz/ready\": dial tcp 10.133.0.167:15021: connect: connection refused"
Oct 30 22:39:17.352599 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:39:17.352566    4473 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-authcfg-http-29f146-openshift-default-67b597f9d5-b5szh"
Oct 30 22:39:17.354224 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:39:17.354199    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-authcfg-http-29f146-openshift-default-67b597f9d5-b5szh"
Oct 30 22:39:20.324034 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:39:20.323998    4473 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fncvg" containerName="extractor"
Oct 30 22:39:50.324777 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:39:50.324739    4473 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fncvg" containerName="extractor"
Oct 30 22:40:20.323340 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:40:20.323264    4473 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fncvg" containerName="extractor"
Oct 30 22:40:42.410522 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:40:42.410484    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-authcfg-http-29f146-6fc88476bc-bnvws"]
Oct 30 22:40:42.521711 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:40:42.521681    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-authcfg-http-29f146-75787ff4dc-wvbh7"]
Oct 30 22:40:47.410816 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:40:47.410766    4473 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-authcfg-http-29f146-6fc88476bc-bnvws" podUID="8e61f8d9-d9f6-4b68-9e6b-434cc58f4f78" containerName="payload-processing" containerID="cri-o://61db328e2d92e41ba8259a31644204807b8322269c40b885caa2ba0f1b0ded6f" gracePeriod=25
Oct 30 22:40:47.522502 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:40:47.522448    4473 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-29f146-75787ff4dc-wvbh7" podUID="42a9e6b6-88e5-477a-8202-512329baa28e" containerName="payload-pre-processing" containerID="cri-o://2c44fb5d5e52ae57ebd405f1ed244ab1b113fe8702e34e29897dfa7ea71cdfab" gracePeriod=25
Oct 30 22:40:47.596893 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:40:47.596863    4473 generic.go:356] "Generic (PLEG): container finished" podID="8e61f8d9-d9f6-4b68-9e6b-434cc58f4f78" containerID="61db328e2d92e41ba8259a31644204807b8322269c40b885caa2ba0f1b0ded6f" exitCode=0
Oct 30 22:40:47.597069 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:40:47.596900    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-authcfg-http-29f146-6fc88476bc-bnvws" event={"ID":"8e61f8d9-d9f6-4b68-9e6b-434cc58f4f78","Type":"ContainerDied","Data":"61db328e2d92e41ba8259a31644204807b8322269c40b885caa2ba0f1b0ded6f"}
Oct 30 22:40:47.671958 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:40:47.671869    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-authcfg-http-29f146-6fc88476bc-bnvws"
Oct 30 22:40:47.754977 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:40:47.754950    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8e61f8d9-d9f6-4b68-9e6b-434cc58f4f78-kube-api-access-fbq69\" (UniqueName: \"kubernetes.io/projected/8e61f8d9-d9f6-4b68-9e6b-434cc58f4f78-kube-api-access-fbq69\") pod \"8e61f8d9-d9f6-4b68-9e6b-434cc58f4f78\" (UID: \"8e61f8d9-d9f6-4b68-9e6b-434cc58f4f78\") "
Oct 30 22:40:47.755108 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:40:47.755031    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8e61f8d9-d9f6-4b68-9e6b-434cc58f4f78-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8e61f8d9-d9f6-4b68-9e6b-434cc58f4f78-plugins-config-volume\") pod \"8e61f8d9-d9f6-4b68-9e6b-434cc58f4f78\" (UID: \"8e61f8d9-d9f6-4b68-9e6b-434cc58f4f78\") "
Oct 30 22:40:47.755394 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:40:47.755371    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8e61f8d9-d9f6-4b68-9e6b-434cc58f4f78-plugins-config-volume" pod "8e61f8d9-d9f6-4b68-9e6b-434cc58f4f78" (UID: "8e61f8d9-d9f6-4b68-9e6b-434cc58f4f78"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:40:47.757284 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:40:47.757262    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8e61f8d9-d9f6-4b68-9e6b-434cc58f4f78-kube-api-access-fbq69" pod "8e61f8d9-d9f6-4b68-9e6b-434cc58f4f78" (UID: "8e61f8d9-d9f6-4b68-9e6b-434cc58f4f78"). InnerVolumeSpecName "kube-api-access-fbq69". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:40:47.772058 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:40:47.772031    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-29f146-75787ff4dc-wvbh7"
Oct 30 22:40:47.856489 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:40:47.856454    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/42a9e6b6-88e5-477a-8202-512329baa28e-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/42a9e6b6-88e5-477a-8202-512329baa28e-plugins-config-volume\") pod \"42a9e6b6-88e5-477a-8202-512329baa28e\" (UID: \"42a9e6b6-88e5-477a-8202-512329baa28e\") "
Oct 30 22:40:47.856733 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:40:47.856502    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/42a9e6b6-88e5-477a-8202-512329baa28e-kube-api-access-j4tj6\" (UniqueName: \"kubernetes.io/projected/42a9e6b6-88e5-477a-8202-512329baa28e-kube-api-access-j4tj6\") pod \"42a9e6b6-88e5-477a-8202-512329baa28e\" (UID: \"42a9e6b6-88e5-477a-8202-512329baa28e\") "
Oct 30 22:40:47.856808 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:40:47.856745    4473 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fbq69\" (UniqueName: \"kubernetes.io/projected/8e61f8d9-d9f6-4b68-9e6b-434cc58f4f78-kube-api-access-fbq69\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:40:47.856808 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:40:47.856759    4473 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8e61f8d9-d9f6-4b68-9e6b-434cc58f4f78-plugins-config-volume\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:40:47.856968 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:40:47.856913    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/42a9e6b6-88e5-477a-8202-512329baa28e-plugins-config-volume" pod "42a9e6b6-88e5-477a-8202-512329baa28e" (UID: "42a9e6b6-88e5-477a-8202-512329baa28e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:40:47.858797 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:40:47.858778    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/42a9e6b6-88e5-477a-8202-512329baa28e-kube-api-access-j4tj6" pod "42a9e6b6-88e5-477a-8202-512329baa28e" (UID: "42a9e6b6-88e5-477a-8202-512329baa28e"). InnerVolumeSpecName "kube-api-access-j4tj6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:40:47.958010 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:40:47.957908    4473 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/42a9e6b6-88e5-477a-8202-512329baa28e-plugins-config-volume\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:40:47.958010 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:40:47.957963    4473 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j4tj6\" (UniqueName: \"kubernetes.io/projected/42a9e6b6-88e5-477a-8202-512329baa28e-kube-api-access-j4tj6\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:40:48.602855 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:40:48.602825    4473 generic.go:356] "Generic (PLEG): container finished" podID="42a9e6b6-88e5-477a-8202-512329baa28e" containerID="2c44fb5d5e52ae57ebd405f1ed244ab1b113fe8702e34e29897dfa7ea71cdfab" exitCode=0
Oct 30 22:40:48.603339 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:40:48.602875    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-29f146-75787ff4dc-wvbh7"
Oct 30 22:40:48.603339 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:40:48.602903    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-29f146-75787ff4dc-wvbh7" event={"ID":"42a9e6b6-88e5-477a-8202-512329baa28e","Type":"ContainerDied","Data":"2c44fb5d5e52ae57ebd405f1ed244ab1b113fe8702e34e29897dfa7ea71cdfab"}
Oct 30 22:40:48.603339 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:40:48.602972    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-29f146-75787ff4dc-wvbh7" event={"ID":"42a9e6b6-88e5-477a-8202-512329baa28e","Type":"ContainerDied","Data":"b80f0a71001e195150019725715cca5892e200bff9a1c10eb93082177b2b5442"}
Oct 30 22:40:48.603339 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:40:48.602994    4473 scope.go:122] "RemoveContainer" containerID="2c44fb5d5e52ae57ebd405f1ed244ab1b113fe8702e34e29897dfa7ea71cdfab"
Oct 30 22:40:48.604528 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:40:48.604510    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-authcfg-http-29f146-6fc88476bc-bnvws"
Oct 30 22:40:48.604615 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:40:48.604539    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-authcfg-http-29f146-6fc88476bc-bnvws" event={"ID":"8e61f8d9-d9f6-4b68-9e6b-434cc58f4f78","Type":"ContainerDied","Data":"589c94acdd6706336f0d4ebd5d8d897e8226255a7b884a1949af178643f89a97"}
Oct 30 22:40:48.612128 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:40:48.612104    4473 scope.go:122] "RemoveContainer" containerID="2c44fb5d5e52ae57ebd405f1ed244ab1b113fe8702e34e29897dfa7ea71cdfab"
Oct 30 22:40:48.612410 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:40:48.612390    4473 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2c44fb5d5e52ae57ebd405f1ed244ab1b113fe8702e34e29897dfa7ea71cdfab\": container with ID starting with 2c44fb5d5e52ae57ebd405f1ed244ab1b113fe8702e34e29897dfa7ea71cdfab not found: ID does not exist" containerID="2c44fb5d5e52ae57ebd405f1ed244ab1b113fe8702e34e29897dfa7ea71cdfab"
Oct 30 22:40:48.612462 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:40:48.612421    4473 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2c44fb5d5e52ae57ebd405f1ed244ab1b113fe8702e34e29897dfa7ea71cdfab"} err="failed to get container status \"2c44fb5d5e52ae57ebd405f1ed244ab1b113fe8702e34e29897dfa7ea71cdfab\": rpc error: code = NotFound desc = could not find container \"2c44fb5d5e52ae57ebd405f1ed244ab1b113fe8702e34e29897dfa7ea71cdfab\": container with ID starting with 2c44fb5d5e52ae57ebd405f1ed244ab1b113fe8702e34e29897dfa7ea71cdfab not found: ID does not exist"
Oct 30 22:40:48.612462 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:40:48.612438    4473 scope.go:122] "RemoveContainer" containerID="61db328e2d92e41ba8259a31644204807b8322269c40b885caa2ba0f1b0ded6f"
Oct 30 22:40:48.771309 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:40:48.771273    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-authcfg-http-29f146-75787ff4dc-wvbh7"]
Oct 30 22:40:48.821139 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:40:48.821098    4473 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-authcfg-http-29f146-75787ff4dc-wvbh7"]
Oct 30 22:40:48.887039 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:40:48.886954    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-authcfg-http-29f146-6fc88476bc-bnvws"]
Oct 30 22:40:48.892909 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:40:48.892881    4473 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-authcfg-http-29f146-6fc88476bc-bnvws"]
Oct 30 22:40:50.324651 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:40:50.324618    4473 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fncvg" containerName="extractor"
Oct 30 22:40:50.356041 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:40:50.356011    4473 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="42a9e6b6-88e5-477a-8202-512329baa28e" path="/var/lib/kubelet/pods/42a9e6b6-88e5-477a-8202-512329baa28e/volumes"
Oct 30 22:40:50.356424 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:40:50.356411    4473 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8e61f8d9-d9f6-4b68-9e6b-434cc58f4f78" path="/var/lib/kubelet/pods/8e61f8d9-d9f6-4b68-9e6b-434cc58f4f78/volumes"
Oct 30 22:40:55.280213 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:40:55.280169    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-authcfg-http-29f146-759775bdb6-gmjg9"]
Oct 30 22:40:55.280719 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:40:55.280435    4473 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-29f146-759775bdb6-gmjg9" podUID="0cde6324-cceb-491c-b40a-50cf2109a542" containerName="maas-api" containerID="cri-o://e35356b3ecfaa58440dd54165a504fa8a5db5a82839a8f47458295fd24504d58" gracePeriod=30
Oct 30 22:40:55.538374 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:40:55.538302    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-29f146-759775bdb6-gmjg9"
Oct 30 22:40:55.636600 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:40:55.636562    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0cde6324-cceb-491c-b40a-50cf2109a542-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0cde6324-cceb-491c-b40a-50cf2109a542-openshift-service-ca\") pod \"0cde6324-cceb-491c-b40a-50cf2109a542\" (UID: \"0cde6324-cceb-491c-b40a-50cf2109a542\") "
Oct 30 22:40:55.636800 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:40:55.636636    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0cde6324-cceb-491c-b40a-50cf2109a542-metrics-certs\" (UniqueName: \"kubernetes.io/secret/0cde6324-cceb-491c-b40a-50cf2109a542-metrics-certs\") pod \"0cde6324-cceb-491c-b40a-50cf2109a542\" (UID: \"0cde6324-cceb-491c-b40a-50cf2109a542\") "
Oct 30 22:40:55.636800 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:40:55.636767    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0cde6324-cceb-491c-b40a-50cf2109a542-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0cde6324-cceb-491c-b40a-50cf2109a542-maas-api-tls\") pod \"0cde6324-cceb-491c-b40a-50cf2109a542\" (UID: \"0cde6324-cceb-491c-b40a-50cf2109a542\") "
Oct 30 22:40:55.636958 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:40:55.636800    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0cde6324-cceb-491c-b40a-50cf2109a542-kube-api-access-dccnd\" (UniqueName: \"kubernetes.io/projected/0cde6324-cceb-491c-b40a-50cf2109a542-kube-api-access-dccnd\") pod \"0cde6324-cceb-491c-b40a-50cf2109a542\" (UID: \"0cde6324-cceb-491c-b40a-50cf2109a542\") "
Oct 30 22:40:55.637028 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:40:55.636965    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0cde6324-cceb-491c-b40a-50cf2109a542-openshift-service-ca" pod "0cde6324-cceb-491c-b40a-50cf2109a542" (UID: "0cde6324-cceb-491c-b40a-50cf2109a542"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:40:55.637237 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:40:55.637216    4473 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0cde6324-cceb-491c-b40a-50cf2109a542-openshift-service-ca\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:40:55.639051 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:40:55.639023    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0cde6324-cceb-491c-b40a-50cf2109a542-metrics-certs" pod "0cde6324-cceb-491c-b40a-50cf2109a542" (UID: "0cde6324-cceb-491c-b40a-50cf2109a542"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:40:55.639196 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:40:55.639174    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0cde6324-cceb-491c-b40a-50cf2109a542-maas-api-tls" pod "0cde6324-cceb-491c-b40a-50cf2109a542" (UID: "0cde6324-cceb-491c-b40a-50cf2109a542"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:40:55.639196 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:40:55.639182    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0cde6324-cceb-491c-b40a-50cf2109a542-kube-api-access-dccnd" pod "0cde6324-cceb-491c-b40a-50cf2109a542" (UID: "0cde6324-cceb-491c-b40a-50cf2109a542"). InnerVolumeSpecName "kube-api-access-dccnd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:40:55.642879 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:40:55.642854    4473 generic.go:356] "Generic (PLEG): container finished" podID="0cde6324-cceb-491c-b40a-50cf2109a542" containerID="e35356b3ecfaa58440dd54165a504fa8a5db5a82839a8f47458295fd24504d58" exitCode=0
Oct 30 22:40:55.643016 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:40:55.642904    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-29f146-759775bdb6-gmjg9"
Oct 30 22:40:55.643085 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:40:55.643052    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-29f146-759775bdb6-gmjg9" event={"ID":"0cde6324-cceb-491c-b40a-50cf2109a542","Type":"ContainerDied","Data":"e35356b3ecfaa58440dd54165a504fa8a5db5a82839a8f47458295fd24504d58"}
Oct 30 22:40:55.643151 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:40:55.643123    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-29f146-759775bdb6-gmjg9" event={"ID":"0cde6324-cceb-491c-b40a-50cf2109a542","Type":"ContainerDied","Data":"df8c6dcce78898d254eda22ca932cbf51aecb6e6101f438aaa128e7da8ca0c70"}
Oct 30 22:40:55.643208 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:40:55.643158    4473 scope.go:122] "RemoveContainer" containerID="e35356b3ecfaa58440dd54165a504fa8a5db5a82839a8f47458295fd24504d58"
Oct 30 22:40:55.670678 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:40:55.670650    4473 scope.go:122] "RemoveContainer" containerID="e35356b3ecfaa58440dd54165a504fa8a5db5a82839a8f47458295fd24504d58"
Oct 30 22:40:55.671462 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:40:55.671310    4473 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e35356b3ecfaa58440dd54165a504fa8a5db5a82839a8f47458295fd24504d58\": container with ID starting with e35356b3ecfaa58440dd54165a504fa8a5db5a82839a8f47458295fd24504d58 not found: ID does not exist" containerID="e35356b3ecfaa58440dd54165a504fa8a5db5a82839a8f47458295fd24504d58"
Oct 30 22:40:55.671462 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:40:55.671417    4473 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e35356b3ecfaa58440dd54165a504fa8a5db5a82839a8f47458295fd24504d58"} err="failed to get container status \"e35356b3ecfaa58440dd54165a504fa8a5db5a82839a8f47458295fd24504d58\": rpc error: code = NotFound desc = could not find container \"e35356b3ecfaa58440dd54165a504fa8a5db5a82839a8f47458295fd24504d58\": container with ID starting with e35356b3ecfaa58440dd54165a504fa8a5db5a82839a8f47458295fd24504d58 not found: ID does not exist"
Oct 30 22:40:55.738313 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:40:55.738272    4473 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0cde6324-cceb-491c-b40a-50cf2109a542-maas-api-tls\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:40:55.738313 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:40:55.738307    4473 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dccnd\" (UniqueName: \"kubernetes.io/projected/0cde6324-cceb-491c-b40a-50cf2109a542-kube-api-access-dccnd\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:40:55.738313 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:40:55.738317    4473 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0cde6324-cceb-491c-b40a-50cf2109a542-metrics-certs\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:40:55.832671 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:40:55.832589    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-authcfg-http-29f146-759775bdb6-gmjg9"]
Oct 30 22:40:55.836476 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:40:55.836445    4473 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-authcfg-http-29f146-759775bdb6-gmjg9"]
Oct 30 22:40:56.355687 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:40:56.355648    4473 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0cde6324-cceb-491c-b40a-50cf2109a542" path="/var/lib/kubelet/pods/0cde6324-cceb-491c-b40a-50cf2109a542/volumes"
Oct 30 22:41:04.128740 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:41:04.128696    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-authcfg-http-29f146-openshift-default-67b597f9d5-b5szh"]
Oct 30 22:41:04.129327 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:41:04.129009    4473 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-authcfg-http-29f146-openshift-default-67b597f9d5-b5szh" podUID="5fcca3b7-5872-4a79-9189-b96f8bf6a6e6" containerName="istio-proxy" containerID="cri-o://7c79f24eebd86c6a4abd049f1a6d8a74430ae8af4532cf7bbb04d736c9dd2adc" gracePeriod=30
Oct 30 22:41:09.415183 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:41:09.415151    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-authcfg-http-29f146-openshift-default-67b597f9d5-b5szh"
Oct 30 22:41:09.582320 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:41:09.582150    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5fcca3b7-5872-4a79-9189-b96f8bf6a6e6-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5fcca3b7-5872-4a79-9189-b96f8bf6a6e6-workload-certs\") pod \"5fcca3b7-5872-4a79-9189-b96f8bf6a6e6\" (UID: \"5fcca3b7-5872-4a79-9189-b96f8bf6a6e6\") "
Oct 30 22:41:09.582320 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:41:09.582218    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5fcca3b7-5872-4a79-9189-b96f8bf6a6e6-kube-api-access-f67f4\" (UniqueName: \"kubernetes.io/projected/5fcca3b7-5872-4a79-9189-b96f8bf6a6e6-kube-api-access-f67f4\") pod \"5fcca3b7-5872-4a79-9189-b96f8bf6a6e6\" (UID: \"5fcca3b7-5872-4a79-9189-b96f8bf6a6e6\") "
Oct 30 22:41:09.582320 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:41:09.582262    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5fcca3b7-5872-4a79-9189-b96f8bf6a6e6-istio-token\" (UniqueName: \"kubernetes.io/projected/5fcca3b7-5872-4a79-9189-b96f8bf6a6e6-istio-token\") pod \"5fcca3b7-5872-4a79-9189-b96f8bf6a6e6\" (UID: \"5fcca3b7-5872-4a79-9189-b96f8bf6a6e6\") "
Oct 30 22:41:09.582320 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:41:09.582304    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/5fcca3b7-5872-4a79-9189-b96f8bf6a6e6-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5fcca3b7-5872-4a79-9189-b96f8bf6a6e6-istio-podinfo\") pod \"5fcca3b7-5872-4a79-9189-b96f8bf6a6e6\" (UID: \"5fcca3b7-5872-4a79-9189-b96f8bf6a6e6\") "
Oct 30 22:41:09.582320 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:41:09.582341    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5fcca3b7-5872-4a79-9189-b96f8bf6a6e6-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5fcca3b7-5872-4a79-9189-b96f8bf6a6e6-istio-data\") pod \"5fcca3b7-5872-4a79-9189-b96f8bf6a6e6\" (UID: \"5fcca3b7-5872-4a79-9189-b96f8bf6a6e6\") "
Oct 30 22:41:09.582882 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:41:09.582420    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5fcca3b7-5872-4a79-9189-b96f8bf6a6e6-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5fcca3b7-5872-4a79-9189-b96f8bf6a6e6-workload-socket\") pod \"5fcca3b7-5872-4a79-9189-b96f8bf6a6e6\" (UID: \"5fcca3b7-5872-4a79-9189-b96f8bf6a6e6\") "
Oct 30 22:41:09.582882 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:41:09.582475    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5fcca3b7-5872-4a79-9189-b96f8bf6a6e6-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5fcca3b7-5872-4a79-9189-b96f8bf6a6e6-credential-socket\") pod \"5fcca3b7-5872-4a79-9189-b96f8bf6a6e6\" (UID: \"5fcca3b7-5872-4a79-9189-b96f8bf6a6e6\") "
Oct 30 22:41:09.582882 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:41:09.582538    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5fcca3b7-5872-4a79-9189-b96f8bf6a6e6-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5fcca3b7-5872-4a79-9189-b96f8bf6a6e6-istiod-ca-cert\") pod \"5fcca3b7-5872-4a79-9189-b96f8bf6a6e6\" (UID: \"5fcca3b7-5872-4a79-9189-b96f8bf6a6e6\") "
Oct 30 22:41:09.582882 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:41:09.582574    4473 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5fcca3b7-5872-4a79-9189-b96f8bf6a6e6-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5fcca3b7-5872-4a79-9189-b96f8bf6a6e6-istio-envoy\") pod \"5fcca3b7-5872-4a79-9189-b96f8bf6a6e6\" (UID: \"5fcca3b7-5872-4a79-9189-b96f8bf6a6e6\") "
Oct 30 22:41:09.590216 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:41:09.586404    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5fcca3b7-5872-4a79-9189-b96f8bf6a6e6-workload-certs" pod "5fcca3b7-5872-4a79-9189-b96f8bf6a6e6" (UID: "5fcca3b7-5872-4a79-9189-b96f8bf6a6e6"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:41:09.590216 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:41:09.587154    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5fcca3b7-5872-4a79-9189-b96f8bf6a6e6-credential-socket" pod "5fcca3b7-5872-4a79-9189-b96f8bf6a6e6" (UID: "5fcca3b7-5872-4a79-9189-b96f8bf6a6e6"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:41:09.590216 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:41:09.587360    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5fcca3b7-5872-4a79-9189-b96f8bf6a6e6-istiod-ca-cert" pod "5fcca3b7-5872-4a79-9189-b96f8bf6a6e6" (UID: "5fcca3b7-5872-4a79-9189-b96f8bf6a6e6"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:41:09.590216 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:41:09.587423    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5fcca3b7-5872-4a79-9189-b96f8bf6a6e6-workload-socket" pod "5fcca3b7-5872-4a79-9189-b96f8bf6a6e6" (UID: "5fcca3b7-5872-4a79-9189-b96f8bf6a6e6"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:41:09.590216 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:41:09.587480    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5fcca3b7-5872-4a79-9189-b96f8bf6a6e6-istio-data" pod "5fcca3b7-5872-4a79-9189-b96f8bf6a6e6" (UID: "5fcca3b7-5872-4a79-9189-b96f8bf6a6e6"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:41:09.590216 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:41:09.588997    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5fcca3b7-5872-4a79-9189-b96f8bf6a6e6-istio-podinfo" pod "5fcca3b7-5872-4a79-9189-b96f8bf6a6e6" (UID: "5fcca3b7-5872-4a79-9189-b96f8bf6a6e6"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:41:09.591218 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:41:09.591178    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5fcca3b7-5872-4a79-9189-b96f8bf6a6e6-istio-token" pod "5fcca3b7-5872-4a79-9189-b96f8bf6a6e6" (UID: "5fcca3b7-5872-4a79-9189-b96f8bf6a6e6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:41:09.591770 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:41:09.591735    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5fcca3b7-5872-4a79-9189-b96f8bf6a6e6-istio-envoy" pod "5fcca3b7-5872-4a79-9189-b96f8bf6a6e6" (UID: "5fcca3b7-5872-4a79-9189-b96f8bf6a6e6"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:41:09.596815 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:41:09.596726    4473 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5fcca3b7-5872-4a79-9189-b96f8bf6a6e6-kube-api-access-f67f4" pod "5fcca3b7-5872-4a79-9189-b96f8bf6a6e6" (UID: "5fcca3b7-5872-4a79-9189-b96f8bf6a6e6"). InnerVolumeSpecName "kube-api-access-f67f4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:41:09.684041 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:41:09.683912    4473 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5fcca3b7-5872-4a79-9189-b96f8bf6a6e6-workload-socket\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:41:09.684041 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:41:09.683979    4473 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5fcca3b7-5872-4a79-9189-b96f8bf6a6e6-credential-socket\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:41:09.684041 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:41:09.683996    4473 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5fcca3b7-5872-4a79-9189-b96f8bf6a6e6-istiod-ca-cert\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:41:09.684041 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:41:09.684010    4473 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5fcca3b7-5872-4a79-9189-b96f8bf6a6e6-istio-envoy\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:41:09.684041 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:41:09.684024    4473 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5fcca3b7-5872-4a79-9189-b96f8bf6a6e6-workload-certs\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:41:09.684041 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:41:09.684041    4473 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f67f4\" (UniqueName: \"kubernetes.io/projected/5fcca3b7-5872-4a79-9189-b96f8bf6a6e6-kube-api-access-f67f4\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:41:09.684541 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:41:09.684056    4473 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5fcca3b7-5872-4a79-9189-b96f8bf6a6e6-istio-token\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:41:09.684541 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:41:09.684071    4473 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5fcca3b7-5872-4a79-9189-b96f8bf6a6e6-istio-podinfo\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:41:09.684541 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:41:09.684085    4473 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5fcca3b7-5872-4a79-9189-b96f8bf6a6e6-istio-data\") on node \"ip-10-0-140-18.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:41:09.724189 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:41:09.724149    4473 generic.go:356] "Generic (PLEG): container finished" podID="5fcca3b7-5872-4a79-9189-b96f8bf6a6e6" containerID="7c79f24eebd86c6a4abd049f1a6d8a74430ae8af4532cf7bbb04d736c9dd2adc" exitCode=0
Oct 30 22:41:09.724394 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:41:09.724217    4473 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-authcfg-http-29f146-openshift-default-67b597f9d5-b5szh"
Oct 30 22:41:09.724394 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:41:09.724214    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-authcfg-http-29f146-openshift-default-67b597f9d5-b5szh" event={"ID":"5fcca3b7-5872-4a79-9189-b96f8bf6a6e6","Type":"ContainerDied","Data":"7c79f24eebd86c6a4abd049f1a6d8a74430ae8af4532cf7bbb04d736c9dd2adc"}
Oct 30 22:41:09.724394 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:41:09.724319    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-authcfg-http-29f146-openshift-default-67b597f9d5-b5szh" event={"ID":"5fcca3b7-5872-4a79-9189-b96f8bf6a6e6","Type":"ContainerDied","Data":"d17642ea1f4a3292d792bd37380a0e27e119aa65d6747c8bf20e82f23f8a8ffa"}
Oct 30 22:41:09.724394 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:41:09.724361    4473 scope.go:122] "RemoveContainer" containerID="7c79f24eebd86c6a4abd049f1a6d8a74430ae8af4532cf7bbb04d736c9dd2adc"
Oct 30 22:41:09.737769 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:41:09.737634    4473 scope.go:122] "RemoveContainer" containerID="fb6077a9a88a59545c393e6ac816011f9795140311ddf45e529ff38b7009853e"
Oct 30 22:41:09.750555 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:41:09.750524    4473 scope.go:122] "RemoveContainer" containerID="7c79f24eebd86c6a4abd049f1a6d8a74430ae8af4532cf7bbb04d736c9dd2adc"
Oct 30 22:41:09.751060 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:41:09.751022    4473 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7c79f24eebd86c6a4abd049f1a6d8a74430ae8af4532cf7bbb04d736c9dd2adc\": container with ID starting with 7c79f24eebd86c6a4abd049f1a6d8a74430ae8af4532cf7bbb04d736c9dd2adc not found: ID does not exist" containerID="7c79f24eebd86c6a4abd049f1a6d8a74430ae8af4532cf7bbb04d736c9dd2adc"
Oct 30 22:41:09.751183 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:41:09.751072    4473 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7c79f24eebd86c6a4abd049f1a6d8a74430ae8af4532cf7bbb04d736c9dd2adc"} err="failed to get container status \"7c79f24eebd86c6a4abd049f1a6d8a74430ae8af4532cf7bbb04d736c9dd2adc\": rpc error: code = NotFound desc = could not find container \"7c79f24eebd86c6a4abd049f1a6d8a74430ae8af4532cf7bbb04d736c9dd2adc\": container with ID starting with 7c79f24eebd86c6a4abd049f1a6d8a74430ae8af4532cf7bbb04d736c9dd2adc not found: ID does not exist"
Oct 30 22:41:09.751183 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:41:09.751098    4473 scope.go:122] "RemoveContainer" containerID="fb6077a9a88a59545c393e6ac816011f9795140311ddf45e529ff38b7009853e"
Oct 30 22:41:09.751485 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:41:09.751449    4473 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fb6077a9a88a59545c393e6ac816011f9795140311ddf45e529ff38b7009853e\": container with ID starting with fb6077a9a88a59545c393e6ac816011f9795140311ddf45e529ff38b7009853e not found: ID does not exist" containerID="fb6077a9a88a59545c393e6ac816011f9795140311ddf45e529ff38b7009853e"
Oct 30 22:41:09.751577 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:41:09.751497    4473 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fb6077a9a88a59545c393e6ac816011f9795140311ddf45e529ff38b7009853e"} err="failed to get container status \"fb6077a9a88a59545c393e6ac816011f9795140311ddf45e529ff38b7009853e\": rpc error: code = NotFound desc = could not find container \"fb6077a9a88a59545c393e6ac816011f9795140311ddf45e529ff38b7009853e\": container with ID starting with fb6077a9a88a59545c393e6ac816011f9795140311ddf45e529ff38b7009853e not found: ID does not exist"
Oct 30 22:41:09.894293 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:41:09.894252    4473 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-authcfg-http-29f146-openshift-default-67b597f9d5-b5szh"]
Oct 30 22:41:09.897646 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:41:09.897611    4473 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-authcfg-http-29f146-openshift-default-67b597f9d5-b5szh"]
Oct 30 22:41:10.375947 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:41:10.374910    4473 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5fcca3b7-5872-4a79-9189-b96f8bf6a6e6" path="/var/lib/kubelet/pods/5fcca3b7-5872-4a79-9189-b96f8bf6a6e6/volumes"
Oct 30 22:41:20.324072 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:41:20.324034    4473 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fncvg" containerName="extractor"
Oct 30 22:41:50.423349 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:41:50.423104    4473 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fncvg" containerName="extractor"
Oct 30 22:42:16.747096 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:42:16.747054    4473 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-kp5pl container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.97:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 22:42:16.747812 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:42:16.747779    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-kp5pl" podUID="2f52b081-50a2-4489-9041-8ef19b475dba" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.97:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:42:16.838318 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:42:16.838273    4473 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-zgjtb container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.98:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 22:42:16.838639 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:42:16.838606    4473 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-zgjtb" podUID="a13c1563-5b04-424a-b761-62a4b85f16e7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.98:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:42:20.323408 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:42:20.323371    4473 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fncvg" containerName="extractor"
Oct 30 22:42:25.397664 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:42:25.397627    4473 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-module-controller-manager-7566cf5fc8-8l7qq_9f6d6982-07b1-44d1-a126-27c73898a66e/manager/0.log"
Oct 30 22:42:26.127560 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:42:26.127528    4473 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-786df48447-pn8gm_6c7988f6-b9bc-4009-8354-73efd639c648/manager/0.log"
Oct 30 22:42:26.953681 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:42:26.953650    4473 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-769875db56-wr8rv_d54f8662-fb93-472e-a930-8d07c7550edb/manager/0.log"
Oct 30 22:42:27.203004 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:42:27.202969    4473 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_usage-logs-tenancy-proxy-9c5d49bcd-9cgjz_8ce72a54-9bb1-47c9-9891-3e998dcf8e2e/proxy/0.log"
Oct 30 22:42:28.241944 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:42:28.241886    4473 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-64979f7667-qkx6z_7c8203e1-57fc-4227-91b6-769ce42af364/maas-api/0.log"
Oct 30 22:42:29.069580 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:42:29.069547    4473 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-7cdbf6dc44-h5d9q_e5b0a336-6057-4b8e-8803-526f1398d0f8/postgres/0.log"
Oct 30 22:42:33.581646 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:42:33.581615    4473 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5cbcf87c76-fnvrz_f1348ad3-7562-42e5-a9fb-5aa36f6eb9c5/kube-auth-proxy/0.log"
Oct 30 22:42:34.489779 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:42:34.489751    4473 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-7bdc57c4cf-5b9r4_1557635a-ac42-4385-99f3-879641eb0ccc/istio-proxy/0.log"
Oct 30 22:42:35.026459 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:42:35.026426    4473 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-7bdc57c4cf-ggdbq_5cbe0443-8a90-4133-9f81-42bdca7d7e1a/istio-proxy/0.log"
Oct 30 22:42:35.297198 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:42:35.297098    4473 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-7bdc57c4cf-kp5pl_2f52b081-50a2-4489-9041-8ef19b475dba/istio-proxy/0.log"
Oct 30 22:42:35.959006 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:42:35.958962    4473 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-7bdc57c4cf-zgjtb_a13c1563-5b04-424a-b761-62a4b85f16e7/istio-proxy/0.log"
Oct 30 22:42:36.486742 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:42:36.486712    4473 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7b98bb678c-jg6ld_540be0d8-cd2b-4397-8be5-3da2d9326eee/payload-pre-processing/0.log"
Oct 30 22:42:36.691200 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:42:36.691170    4473 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-55fbbbf9dc-8vvh2_61fae073-2a6c-422c-9e51-26cdaff5da9f/payload-processing/0.log"
Oct 30 22:42:38.043744 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:42:38.043711    4473 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-cbd4f5586-5g7dx_da982b32-3d1a-4766-b9d4-6804b0684cd3/main/0.log"
Oct 30 22:42:38.554658 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:42:38.554623    4473 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-embedding-simulated-kserve-67bf594cd6-mgkz8_d3644b22-59fe-41ab-99b5-227e15669a15/main/0.log"
Oct 30 22:42:38.819004 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:42:38.818930    4473 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-69b588897d-fkv7q_8265cdcf-eaa9-4e4f-8250-915df95739c2/main/0.log"
Oct 30 22:42:39.279346 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:42:39.279315    4473 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_echo-simulator-kserve-64b6f8f96c-m6tlf_e9ca7766-41cc-4042-ae52-ce6451f31d4b/main/0.log"
Oct 30 22:42:50.296122 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:42:50.296072    4473 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-module-controller-manager-7566cf5fc8-8l7qq_9f6d6982-07b1-44d1-a126-27c73898a66e/manager/0.log"
Oct 30 22:42:50.323905 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:42:50.323868    4473 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fncvg" containerName="extractor"
Oct 30 22:42:51.020549 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:42:51.020519    4473 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-786df48447-pn8gm_6c7988f6-b9bc-4009-8354-73efd639c648/manager/0.log"
Oct 30 22:42:51.591336 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:42:51.591308    4473 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-769875db56-wr8rv_d54f8662-fb93-472e-a930-8d07c7550edb/manager/0.log"
Oct 30 22:42:51.863904 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:42:51.863877    4473 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_usage-logs-tenancy-proxy-9c5d49bcd-9cgjz_8ce72a54-9bb1-47c9-9891-3e998dcf8e2e/proxy/0.log"
Oct 30 22:42:52.701559 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:42:52.701532    4473 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-64979f7667-qkx6z_7c8203e1-57fc-4227-91b6-769ce42af364/maas-api/0.log"
Oct 30 22:42:53.538935 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:42:53.538890    4473 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-7cdbf6dc44-h5d9q_e5b0a336-6057-4b8e-8803-526f1398d0f8/postgres/0.log"
Oct 30 22:42:57.471729 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:42:57.471696    4473 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5cbcf87c76-fnvrz_f1348ad3-7562-42e5-a9fb-5aa36f6eb9c5/kube-auth-proxy/0.log"
Oct 30 22:42:58.341216 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:42:58.341180    4473 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-7bdc57c4cf-5b9r4_1557635a-ac42-4385-99f3-879641eb0ccc/istio-proxy/0.log"
Oct 30 22:42:58.858595 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:42:58.858565    4473 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-7bdc57c4cf-ggdbq_5cbe0443-8a90-4133-9f81-42bdca7d7e1a/istio-proxy/0.log"
Oct 30 22:42:59.129264 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:42:59.129228    4473 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-7bdc57c4cf-kp5pl_2f52b081-50a2-4489-9041-8ef19b475dba/istio-proxy/0.log"
Oct 30 22:42:59.766777 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:42:59.766735    4473 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-7bdc57c4cf-zgjtb_a13c1563-5b04-424a-b761-62a4b85f16e7/istio-proxy/0.log"
Oct 30 22:43:00.199548 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:43:00.199510    4473 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7b98bb678c-jg6ld_540be0d8-cd2b-4397-8be5-3da2d9326eee/payload-pre-processing/0.log"
Oct 30 22:43:00.401131 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:43:00.401098    4473 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-55fbbbf9dc-8vvh2_61fae073-2a6c-422c-9e51-26cdaff5da9f/payload-processing/0.log"
Oct 30 22:43:01.205156 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:43:01.205123    4473 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-cbd4f5586-5g7dx_da982b32-3d1a-4766-b9d4-6804b0684cd3/main/0.log"
Oct 30 22:43:01.742332 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:43:01.742296    4473 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-embedding-simulated-kserve-67bf594cd6-mgkz8_d3644b22-59fe-41ab-99b5-227e15669a15/main/0.log"
Oct 30 22:43:02.011526 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:43:02.011446    4473 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-69b588897d-fkv7q_8265cdcf-eaa9-4e4f-8250-915df95739c2/main/0.log"
Oct 30 22:43:02.482763 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:43:02.482727    4473 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_echo-simulator-kserve-64b6f8f96c-m6tlf_e9ca7766-41cc-4042-ae52-ce6451f31d4b/main/0.log"
Oct 30 22:43:20.323171 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:43:20.323075    4473 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fncvg" containerName="extractor"
Oct 30 22:43:31.622038 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:43:31.622004    4473 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-786df48447-pn8gm_6c7988f6-b9bc-4009-8354-73efd639c648/manager/0.log"
Oct 30 22:43:32.198695 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:43:32.198653    4473 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-786df48447-pn8gm_6c7988f6-b9bc-4009-8354-73efd639c648/manager/0.log"
Oct 30 22:43:32.890417 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:43:32.890386    4473 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-55fbbbf9dc-8vvh2_61fae073-2a6c-422c-9e51-26cdaff5da9f/payload-processing/0.log"
Oct 30 22:43:33.122033 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:43:33.122007    4473 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-55fbbbf9dc-8vvh2_61fae073-2a6c-422c-9e51-26cdaff5da9f/payload-processing/0.log"
Oct 30 22:43:50.324966 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:43:50.324907    4473 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fncvg" containerName="extractor"
Oct 30 22:44:10.165180 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:44:10.165115    4473 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-786df48447-pn8gm_6c7988f6-b9bc-4009-8354-73efd639c648/manager/0.log"
Oct 30 22:44:10.743860 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:44:10.743823    4473 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-786df48447-pn8gm_6c7988f6-b9bc-4009-8354-73efd639c648/manager/0.log"
Oct 30 22:44:11.454500 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:44:11.454473    4473 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-55fbbbf9dc-8vvh2_61fae073-2a6c-422c-9e51-26cdaff5da9f/payload-processing/0.log"
Oct 30 22:44:11.660301 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:44:11.660272    4473 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-55fbbbf9dc-8vvh2_61fae073-2a6c-422c-9e51-26cdaff5da9f/payload-processing/0.log"
Oct 30 22:44:12.017207 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:44:12.017166    4473 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-must-gather-mff8v/must-gather-mn6kc"]
Oct 30 22:44:12.017796 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:44:12.017778    4473 cpu_manager.go:414] "containerMap: removing container" podUID="0cde6324-cceb-491c-b40a-50cf2109a542" containerName="maas-api"
Oct 30 22:44:12.017846 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:44:12.017799    4473 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0cde6324-cceb-491c-b40a-50cf2109a542" containerName="maas-api"
Oct 30 22:44:12.017846 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:44:12.017834    4473 cpu_manager.go:414] "containerMap: removing container" podUID="8e61f8d9-d9f6-4b68-9e6b-434cc58f4f78" containerName="payload-processing"
Oct 30 22:44:12.017846 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:44:12.017842    4473 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8e61f8d9-d9f6-4b68-9e6b-434cc58f4f78" containerName="payload-processing"
Oct 30 22:44:12.017981 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:44:12.017860    4473 cpu_manager.go:414] "containerMap: removing container" podUID="5fcca3b7-5872-4a79-9189-b96f8bf6a6e6" containerName="istio-proxy"
Oct 30 22:44:12.017981 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:44:12.017870    4473 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5fcca3b7-5872-4a79-9189-b96f8bf6a6e6" containerName="istio-proxy"
Oct 30 22:44:12.017981 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:44:12.017888    4473 cpu_manager.go:414] "containerMap: removing container" podUID="5fcca3b7-5872-4a79-9189-b96f8bf6a6e6" containerName="istio-proxy"
Oct 30 22:44:12.017981 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:44:12.017904    4473 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5fcca3b7-5872-4a79-9189-b96f8bf6a6e6" containerName="istio-proxy"
Oct 30 22:44:12.017981 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:44:12.017937    4473 cpu_manager.go:414] "containerMap: removing container" podUID="42a9e6b6-88e5-477a-8202-512329baa28e" containerName="payload-pre-processing"
Oct 30 22:44:12.017981 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:44:12.017944    4473 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="42a9e6b6-88e5-477a-8202-512329baa28e" containerName="payload-pre-processing"
Oct 30 22:44:12.018174 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:44:12.018044    4473 memory_manager.go:365] "RemoveStaleState removing state" podUID="8e61f8d9-d9f6-4b68-9e6b-434cc58f4f78" containerName="payload-processing"
Oct 30 22:44:12.018174 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:44:12.018060    4473 memory_manager.go:365] "RemoveStaleState removing state" podUID="5fcca3b7-5872-4a79-9189-b96f8bf6a6e6" containerName="istio-proxy"
Oct 30 22:44:12.018174 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:44:12.018069    4473 memory_manager.go:365] "RemoveStaleState removing state" podUID="5fcca3b7-5872-4a79-9189-b96f8bf6a6e6" containerName="istio-proxy"
Oct 30 22:44:12.018174 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:44:12.018078    4473 memory_manager.go:365] "RemoveStaleState removing state" podUID="42a9e6b6-88e5-477a-8202-512329baa28e" containerName="payload-pre-processing"
Oct 30 22:44:12.018174 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:44:12.018092    4473 memory_manager.go:365] "RemoveStaleState removing state" podUID="0cde6324-cceb-491c-b40a-50cf2109a542" containerName="maas-api"
Oct 30 22:44:12.021642 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:44:12.021616    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-mff8v/must-gather-mn6kc"
Oct 30 22:44:12.074649 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:44:12.074612    4473 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-mff8v\"/\"kube-root-ca.crt\""
Oct 30 22:44:12.125327 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:44:12.125286    4473 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-mff8v\"/\"openshift-service-ca.crt\""
Oct 30 22:44:12.125533 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:44:12.125336    4473 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-mff8v\"/\"default-dockercfg-nl99d\""
Oct 30 22:44:12.127350 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:44:12.127324    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-mff8v/must-gather-mn6kc"]
Oct 30 22:44:12.177270 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:44:12.177228    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6xkrf\" (UniqueName: \"kubernetes.io/projected/f2768a13-94b4-4480-92c3-56045c577755-kube-api-access-6xkrf\") pod \"must-gather-mn6kc\" (UID: \"f2768a13-94b4-4480-92c3-56045c577755\") " pod="openshift-must-gather-mff8v/must-gather-mn6kc"
Oct 30 22:44:12.177486 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:44:12.177279    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/f2768a13-94b4-4480-92c3-56045c577755-must-gather-output\") pod \"must-gather-mn6kc\" (UID: \"f2768a13-94b4-4480-92c3-56045c577755\") " pod="openshift-must-gather-mff8v/must-gather-mn6kc"
Oct 30 22:44:12.278854 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:44:12.278756    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6xkrf\" (UniqueName: \"kubernetes.io/projected/f2768a13-94b4-4480-92c3-56045c577755-kube-api-access-6xkrf\") pod \"must-gather-mn6kc\" (UID: \"f2768a13-94b4-4480-92c3-56045c577755\") " pod="openshift-must-gather-mff8v/must-gather-mn6kc"
Oct 30 22:44:12.278854 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:44:12.278812    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/f2768a13-94b4-4480-92c3-56045c577755-must-gather-output\") pod \"must-gather-mn6kc\" (UID: \"f2768a13-94b4-4480-92c3-56045c577755\") " pod="openshift-must-gather-mff8v/must-gather-mn6kc"
Oct 30 22:44:12.279237 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:44:12.279215    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/f2768a13-94b4-4480-92c3-56045c577755-must-gather-output\") pod \"must-gather-mn6kc\" (UID: \"f2768a13-94b4-4480-92c3-56045c577755\") " pod="openshift-must-gather-mff8v/must-gather-mn6kc"
Oct 30 22:44:12.342757 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:44:12.342715    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6xkrf\" (UniqueName: \"kubernetes.io/projected/f2768a13-94b4-4480-92c3-56045c577755-kube-api-access-6xkrf\") pod \"must-gather-mn6kc\" (UID: \"f2768a13-94b4-4480-92c3-56045c577755\") " pod="openshift-must-gather-mff8v/must-gather-mn6kc"
Oct 30 22:44:12.352079 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:44:12.352047    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-mff8v/must-gather-mn6kc"
Oct 30 22:44:12.729812 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:44:12.729783    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-mff8v/must-gather-mn6kc"]
Oct 30 22:44:12.731731 ip-10-0-140-18 kubenswrapper[4473]: W1030 22:44:12.731698    4473 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf2768a13_94b4_4480_92c3_56045c577755.slice/crio-46f1fa236041b025e43c37d53b576e85cf7e4adcf4d83ecfc2f441fcf96a89fa WatchSource:0}: Error finding container 46f1fa236041b025e43c37d53b576e85cf7e4adcf4d83ecfc2f441fcf96a89fa: Status 404 returned error can't find the container with id 46f1fa236041b025e43c37d53b576e85cf7e4adcf4d83ecfc2f441fcf96a89fa
Oct 30 22:44:12.733520 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:44:12.733502    4473 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 22:44:13.738438 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:44:13.738399    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-mff8v/must-gather-mn6kc" event={"ID":"f2768a13-94b4-4480-92c3-56045c577755","Type":"ContainerStarted","Data":"46f1fa236041b025e43c37d53b576e85cf7e4adcf4d83ecfc2f441fcf96a89fa"}
Oct 30 22:44:14.745044 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:44:14.745008    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-mff8v/must-gather-mn6kc" event={"ID":"f2768a13-94b4-4480-92c3-56045c577755","Type":"ContainerStarted","Data":"84c32ceb3e1f2d64e46a298d1ab3add33260e4503034d62c92ddb1dc9d42bdb2"}
Oct 30 22:44:14.745504 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:44:14.745055    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-mff8v/must-gather-mn6kc" event={"ID":"f2768a13-94b4-4480-92c3-56045c577755","Type":"ContainerStarted","Data":"d2882380322100807fe5143b91944f00b7d85a84649b7177f460049634a6f664"}
Oct 30 22:44:14.879761 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:44:14.879709    4473 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-must-gather-mff8v/must-gather-mn6kc" podStartSLOduration=2.546034787 podStartE2EDuration="3.879694177s" totalImagesPullingTime="1.33365939s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:44:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:44:14.879060939 +0000 UTC m=+3700.652429427" watchObservedRunningTime="2026-10-30 22:44:14.879694177 +0000 UTC m=+3700.653062664"
Oct 30 22:44:20.327670 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:44:20.327631    4473 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fncvg" containerName="extractor"
Oct 30 22:44:21.888942 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:44:21.888557    4473 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-dt7cq_0032cb99-3a37-40c6-98cb-2d40ae4a9319/global-pull-secret-syncer/0.log"
Oct 30 22:44:24.853944 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:44:24.849314    4473 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-krcxf_06552eb8-7379-40df-b31c-a133e646afb9/konnectivity-agent/0.log"
Oct 30 22:44:26.575807 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:44:26.575769    4473 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-140-18.us-west-1.compute.internal_aae6ff4840f4ffd5842f280d0afa74f6/haproxy/0.log"
Oct 30 22:44:31.496823 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:44:31.496789    4473 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-must-gather-mff8v/perf-node-gather-daemonset-8sfrj"]
Oct 30 22:44:31.504432 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:44:31.504404    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-mff8v/perf-node-gather-daemonset-8sfrj"
Oct 30 22:44:31.558443 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:44:31.558410    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a80346cb-82e7-455d-b384-ac91d952e8f2-proc\") pod \"perf-node-gather-daemonset-8sfrj\" (UID: \"a80346cb-82e7-455d-b384-ac91d952e8f2\") " pod="openshift-must-gather-mff8v/perf-node-gather-daemonset-8sfrj"
Oct 30 22:44:31.558639 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:44:31.558535    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a80346cb-82e7-455d-b384-ac91d952e8f2-lib-modules\") pod \"perf-node-gather-daemonset-8sfrj\" (UID: \"a80346cb-82e7-455d-b384-ac91d952e8f2\") " pod="openshift-must-gather-mff8v/perf-node-gather-daemonset-8sfrj"
Oct 30 22:44:31.558639 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:44:31.558573    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a80346cb-82e7-455d-b384-ac91d952e8f2-podres\") pod \"perf-node-gather-daemonset-8sfrj\" (UID: \"a80346cb-82e7-455d-b384-ac91d952e8f2\") " pod="openshift-must-gather-mff8v/perf-node-gather-daemonset-8sfrj"
Oct 30 22:44:31.558776 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:44:31.558659    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-76qzp\" (UniqueName: \"kubernetes.io/projected/a80346cb-82e7-455d-b384-ac91d952e8f2-kube-api-access-76qzp\") pod \"perf-node-gather-daemonset-8sfrj\" (UID: \"a80346cb-82e7-455d-b384-ac91d952e8f2\") " pod="openshift-must-gather-mff8v/perf-node-gather-daemonset-8sfrj"
Oct 30 22:44:31.558776 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:44:31.558708    4473 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a80346cb-82e7-455d-b384-ac91d952e8f2-sys\") pod \"perf-node-gather-daemonset-8sfrj\" (UID: \"a80346cb-82e7-455d-b384-ac91d952e8f2\") " pod="openshift-must-gather-mff8v/perf-node-gather-daemonset-8sfrj"
Oct 30 22:44:31.600987 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:44:31.600947    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-mff8v/perf-node-gather-daemonset-8sfrj"]
Oct 30 22:44:31.659285 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:44:31.659246    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-76qzp\" (UniqueName: \"kubernetes.io/projected/a80346cb-82e7-455d-b384-ac91d952e8f2-kube-api-access-76qzp\") pod \"perf-node-gather-daemonset-8sfrj\" (UID: \"a80346cb-82e7-455d-b384-ac91d952e8f2\") " pod="openshift-must-gather-mff8v/perf-node-gather-daemonset-8sfrj"
Oct 30 22:44:31.659486 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:44:31.659332    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a80346cb-82e7-455d-b384-ac91d952e8f2-sys\") pod \"perf-node-gather-daemonset-8sfrj\" (UID: \"a80346cb-82e7-455d-b384-ac91d952e8f2\") " pod="openshift-must-gather-mff8v/perf-node-gather-daemonset-8sfrj"
Oct 30 22:44:31.659486 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:44:31.659392    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a80346cb-82e7-455d-b384-ac91d952e8f2-proc\") pod \"perf-node-gather-daemonset-8sfrj\" (UID: \"a80346cb-82e7-455d-b384-ac91d952e8f2\") " pod="openshift-must-gather-mff8v/perf-node-gather-daemonset-8sfrj"
Oct 30 22:44:31.659486 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:44:31.659467    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a80346cb-82e7-455d-b384-ac91d952e8f2-lib-modules\") pod \"perf-node-gather-daemonset-8sfrj\" (UID: \"a80346cb-82e7-455d-b384-ac91d952e8f2\") " pod="openshift-must-gather-mff8v/perf-node-gather-daemonset-8sfrj"
Oct 30 22:44:31.659664 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:44:31.659485    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a80346cb-82e7-455d-b384-ac91d952e8f2-sys\") pod \"perf-node-gather-daemonset-8sfrj\" (UID: \"a80346cb-82e7-455d-b384-ac91d952e8f2\") " pod="openshift-must-gather-mff8v/perf-node-gather-daemonset-8sfrj"
Oct 30 22:44:31.659664 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:44:31.659494    4473 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a80346cb-82e7-455d-b384-ac91d952e8f2-podres\") pod \"perf-node-gather-daemonset-8sfrj\" (UID: \"a80346cb-82e7-455d-b384-ac91d952e8f2\") " pod="openshift-must-gather-mff8v/perf-node-gather-daemonset-8sfrj"
Oct 30 22:44:31.659664 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:44:31.659552    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a80346cb-82e7-455d-b384-ac91d952e8f2-podres\") pod \"perf-node-gather-daemonset-8sfrj\" (UID: \"a80346cb-82e7-455d-b384-ac91d952e8f2\") " pod="openshift-must-gather-mff8v/perf-node-gather-daemonset-8sfrj"
Oct 30 22:44:31.659664 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:44:31.659589    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a80346cb-82e7-455d-b384-ac91d952e8f2-proc\") pod \"perf-node-gather-daemonset-8sfrj\" (UID: \"a80346cb-82e7-455d-b384-ac91d952e8f2\") " pod="openshift-must-gather-mff8v/perf-node-gather-daemonset-8sfrj"
Oct 30 22:44:31.659664 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:44:31.659616    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a80346cb-82e7-455d-b384-ac91d952e8f2-lib-modules\") pod \"perf-node-gather-daemonset-8sfrj\" (UID: \"a80346cb-82e7-455d-b384-ac91d952e8f2\") " pod="openshift-must-gather-mff8v/perf-node-gather-daemonset-8sfrj"
Oct 30 22:44:31.720727 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:44:31.720697    4473 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-76qzp\" (UniqueName: \"kubernetes.io/projected/a80346cb-82e7-455d-b384-ac91d952e8f2-kube-api-access-76qzp\") pod \"perf-node-gather-daemonset-8sfrj\" (UID: \"a80346cb-82e7-455d-b384-ac91d952e8f2\") " pod="openshift-must-gather-mff8v/perf-node-gather-daemonset-8sfrj"
Oct 30 22:44:31.818355 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:44:31.818265    4473 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-mff8v/perf-node-gather-daemonset-8sfrj"
Oct 30 22:44:32.204047 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:44:32.204019    4473 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-mff8v/perf-node-gather-daemonset-8sfrj"]
Oct 30 22:44:32.206446 ip-10-0-140-18 kubenswrapper[4473]: W1030 22:44:32.206421    4473 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-poda80346cb_82e7_455d_b384_ac91d952e8f2.slice/crio-f8983e5293cde8c8ea49e77c8aad40e069a92873252bd9d4aff7654d750dd541 WatchSource:0}: Error finding container f8983e5293cde8c8ea49e77c8aad40e069a92873252bd9d4aff7654d750dd541: Status 404 returned error can't find the container with id f8983e5293cde8c8ea49e77c8aad40e069a92873252bd9d4aff7654d750dd541
Oct 30 22:44:32.874430 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:44:32.874393    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-mff8v/perf-node-gather-daemonset-8sfrj" event={"ID":"a80346cb-82e7-455d-b384-ac91d952e8f2","Type":"ContainerStarted","Data":"b567a99385552810390157e701f2c6189e2fd4051a16464f39c1fe908bff9ad2"}
Oct 30 22:44:32.874939 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:44:32.874449    4473 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-mff8v/perf-node-gather-daemonset-8sfrj" event={"ID":"a80346cb-82e7-455d-b384-ac91d952e8f2","Type":"ContainerStarted","Data":"f8983e5293cde8c8ea49e77c8aad40e069a92873252bd9d4aff7654d750dd541"}
Oct 30 22:44:32.874939 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:44:32.874523    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-mff8v/perf-node-gather-daemonset-8sfrj"
Oct 30 22:44:32.991414 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:44:32.991364    4473 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-must-gather-mff8v/perf-node-gather-daemonset-8sfrj" podStartSLOduration=1.991345613 podStartE2EDuration="1.991345613s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 22:44:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:44:32.990586065 +0000 UTC m=+3718.763954552" watchObservedRunningTime="2026-10-30 22:44:32.991345613 +0000 UTC m=+3718.764714100"
Oct 30 22:44:36.507339 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:44:36.507293    4473 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f3a5f77e-d987-4fec-90f4-88ca2f281d0e/alertmanager/0.log"
Oct 30 22:44:36.928963 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:44:36.928901    4473 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f3a5f77e-d987-4fec-90f4-88ca2f281d0e/config-reloader/0.log"
Oct 30 22:44:37.348032 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:44:37.347912    4473 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f3a5f77e-d987-4fec-90f4-88ca2f281d0e/kube-rbac-proxy-web/0.log"
Oct 30 22:44:37.765953 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:44:37.765888    4473 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f3a5f77e-d987-4fec-90f4-88ca2f281d0e/kube-rbac-proxy/0.log"
Oct 30 22:44:38.188479 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:44:38.188437    4473 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f3a5f77e-d987-4fec-90f4-88ca2f281d0e/kube-rbac-proxy-metric/0.log"
Oct 30 22:44:38.608836 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:44:38.608755    4473 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f3a5f77e-d987-4fec-90f4-88ca2f281d0e/prom-label-proxy/0.log"
Oct 30 22:44:38.891828 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:44:38.891798    4473 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-mff8v/perf-node-gather-daemonset-8sfrj"
Oct 30 22:44:39.041064 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:44:39.041024    4473 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f3a5f77e-d987-4fec-90f4-88ca2f281d0e/init-config-reloader/0.log"
Oct 30 22:44:40.011349 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:44:40.011312    4473 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-5dc8d7d89b-b2pn2_a1c5b288-d0e3-4e50-a9eb-f2d2660b3718/kube-state-metrics/0.log"
Oct 30 22:44:40.421086 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:44:40.421058    4473 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-5dc8d7d89b-b2pn2_a1c5b288-d0e3-4e50-a9eb-f2d2660b3718/kube-rbac-proxy-main/0.log"
Oct 30 22:44:40.839631 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:44:40.839547    4473 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-5dc8d7d89b-b2pn2_a1c5b288-d0e3-4e50-a9eb-f2d2660b3718/kube-rbac-proxy-self/0.log"
Oct 30 22:44:41.686266 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:44:41.686238    4473 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-77f4f76594-jjbhq_ceb2c618-a371-489e-b148-1ae43d54a3d3/monitoring-plugin/0.log"
Oct 30 22:44:42.113094 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:44:42.113021    4473 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-kzr98_a20fdb9f-75a2-4373-9337-72fa18ffef81/node-exporter/0.log"
Oct 30 22:44:42.532617 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:44:42.532586    4473 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-kzr98_a20fdb9f-75a2-4373-9337-72fa18ffef81/kube-rbac-proxy/0.log"
Oct 30 22:44:42.950480 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:44:42.950453    4473 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-kzr98_a20fdb9f-75a2-4373-9337-72fa18ffef81/init-textfile/0.log"
Oct 30 22:44:50.324167 ip-10-0-140-18 kubenswrapper[4473]: E1030 22:44:50.324084    4473 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fncvg" containerName="extractor"
Oct 30 22:44:52.683057 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:44:52.683022    4473 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-68886bc5bd-pbpz9_dd17e003-7b46-4f0d-b4e6-fad470cb81e8/thanos-query/0.log"
Oct 30 22:44:53.105366 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:44:53.105278    4473 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-68886bc5bd-pbpz9_dd17e003-7b46-4f0d-b4e6-fad470cb81e8/kube-rbac-proxy-web/0.log"
Oct 30 22:44:53.533139 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:44:53.533100    4473 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-68886bc5bd-pbpz9_dd17e003-7b46-4f0d-b4e6-fad470cb81e8/kube-rbac-proxy/0.log"
Oct 30 22:44:53.951477 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:44:53.951448    4473 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-68886bc5bd-pbpz9_dd17e003-7b46-4f0d-b4e6-fad470cb81e8/prom-label-proxy/0.log"
Oct 30 22:44:54.370808 ip-10-0-140-18 kubenswrapper[4473]: I1030 22:44:54.370768    4473 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-68886bc5bd-pbpz9_dd17e003-7b46-4f0d-b4e6-fad470cb81e8/kube-rbac-proxy-rules/0.log"